The Kuwahara filter is a non-linear smoothing filter used in image processing for adaptive noise reduction. Most filters that are used for image smoothing are linear low-pass filters that effectively reduce noise but also blur out the edges. However the Kuwahara filter is able to apply smoothing on the image while preserving the edges. It is named after Michiyoshi Kuwahara, Ph.D., who worked at Kyoto and Osaka Sangyo Universities in Japan, developing early medical imaging of dynamic heart muscle in the 1970s and 80s. == The Kuwahara operator == Suppose that I ( x , y ) {\displaystyle I(x,y)} is a grey scale image and that we take a square window of size 2 a + 1 {\displaystyle 2a+1} centered around a point ( x , y ) {\displaystyle (x,y)} in the image. This square can be divided into four smaller square regions Q i = 1 ⋯ 4 {\displaystyle Q_{i=1\cdots 4}} each of which will be Q i ( x , y ) = { [ x , x + a ] × [ y , y + a ] if i = 1 [ x − a , x ] × [ y , y + a ] if i = 2 [ x − a , x ] × [ y − a , y ] if i = 3 [ x , x + a ] × [ y − a , y ] if i = 4 {\displaystyle Q_{i}(x,y)={\begin{cases}\left[x,x+a\right]\times \left[y,y+a\right]&{\mbox{ if }}i=1\\\left[x-a,x\right]\times \left[y,y+a\right]&{\mbox{ if }}i=2\\\left[x-a,x\right]\times \left[y-a,y\right]&{\mbox{ if }}i=3\\\left[x,x+a\right]\times \left[y-a,y\right]&{\mbox{ if }}i=4\\\end{cases}}} where × {\displaystyle \times } is the cartesian product. Pixels located on the borders between two regions belong to both regions so there is a slight overlap between subregions. The arithmetic mean m i ( x , y ) {\displaystyle m_{i}(x,y)} and standard deviation σ i ( x , y ) {\displaystyle \sigma _{i}(x,y)} of the four regions centered around a pixel (x,y) are calculated and used to determine the value of the central pixel. The output of the Kuwahara filter Φ ( x , y ) {\displaystyle \Phi (x,y)} for any point ( x , y ) {\displaystyle (x,y)} is then given by Φ ( x , y ) = m i ( x , y ) {\textstyle \Phi (x,y)=m_{i}(x,y)} where i = a r g min j σ j ( x , y ) {\displaystyle i=\operatorname {arg\min } _{j}\sigma _{j}(x,y)} . This means that the central pixel will take the mean value of the area that is most homogenous. The location of the pixel in relation to an edge plays a great role in determining which region will have the greater standard deviation. If for example the pixel is located on a dark side of an edge it will most probably take the mean value of the dark region. On the other hand, should the pixel be on the lighter side of an edge it will most probably take a light value. On the event that the pixel is located on the edge it will take the value of the more smooth, least textured region. The fact that the filter takes into account the homogeneity of the regions ensures that it will preserve the edges while using the mean creates the blurring effect. Similarly to the median filter, the Kuwahara filter uses a sliding window approach to access every pixel in the image. The size of the window is chosen in advance and may vary depending on the desired level of blur in the final image. Bigger windows typically result in the creation of more abstract images whereas small windows produce images that retain their detail. Typically windows are chosen to be square with sides that have an odd number of pixels for symmetry. However, there are variations of the Kuwahara filter that use rectangular windows. Additionally, the subregions do not need to overlap or have the same size as long as they cover all of the window. == Color images == For color images, the filter should not be performed by applying the filter to each RGB channel separately, and then recombining the three filtered color channels to form the filtered RGB image. The main problem with that is that the quadrants will have different standard deviations for each of the channels. For example, the upper left quadrant may have the lowest standard deviation in the red channel, but the lower right quadrant may have the lowest standard deviation in the green channel. This situation would result in the color of the central pixel to be determined by different regions, which might result in color artifacts or blurrier edges. To overcome this problem, for color images a slightly modified Kuwahara filter must be used. The image is first converted into another color space, the HSV color space. The modified filter then operates on only the "brightness" channel, the Value coordinate in the HSV model. The variance of the "brightness" of each quadrant is calculated to determine the quadrant from which the final filtered color should be taken from. The filter will produce an output for each channel which will correspond to the mean of that channel from the quadrant that had the lowest standard deviation in "brightness". This ensures that only one region will determine the RGB values of the central pixel. ImageMagick uses a similar approach, but using the Rec. 709 Luma as the brightness metric. === Julia Implementation === == Applications == Originally the Kuwahara filter was proposed for use in processing RI-angiocardiographic images of the cardiovascular system. The fact that any edges are preserved when smoothing makes it especially useful for feature extraction and segmentation and explains why it is used in medical imaging. The Kuwahara filter however also finds many applications in artistic imaging and fine-art photography due to its ability to remove textures and sharpen the edges of photographs. The level of abstraction helps create a desirable painting-like effect in artistic photographs especially in the case of the colored image version of the filter. These applications have known great success and have encouraged similar research in the field of image processing for the arts. Although the vast majority of applications have been in the field of image processing there have been cases that use modifications of the Kuwahara filter for machine learning tasks such as clustering. The Kuwahara filter has been implemented in CVIPtools. The Kuwahara filter is present as a shader node in Blender. == Drawbacks and restrictions == The Kuwahara filter despite its capabilities in edge preservation has certain drawbacks. At a first glance it is noticeable that the Kuwahara filter does not take into account the case where two regions have equal standard deviations. This is not often the case in real images since it is rather hard to find two regions with exactly the same standard deviation due to the noise that is always present. In cases where two regions have similar standard deviations the value of the center pixel could be decided at random by the noise in these regions. Again this would not be a problem if the regions had the same mean. However, it is not unusual for regions of very different means to have the same standard deviation. This makes the Kuwahara filter susceptible to noise. Different ways have been proposed for dealing with this issue, one of which is to set the value of the center pixel to ( m 1 + m 2 ) / 2 {\textstyle (m_{1}+m_{2})/2} in cases where the standard deviation of two regions do not differ more than a certain value D {\displaystyle D} . The Kuwahara filter is also known to create block artifacts in the images especially in regions of the image that are highly textured. These blocks disrupt the smoothness of the image and are considered to have a negative effect in the aesthetics of the image. This phenomenon occurs due to the division of the window into square regions. A way to overcome this effect is to take windows that are not rectangular(i.e. circular windows) and separate them into more non-rectangular regions. There have also been approaches where the filter adapts its window depending on the input image. == Extensions of the Kuwahara filter == The success of the Kuwahara filter has spurred an increase the development of edge-enhancing smoothing filters. Several variations have been proposed for similar use most of which attempt to deal with the drawbacks of the original Kuwahara filter. The "Generalized Kuwahara filter" proposed by P. Bakker considers several windows that contain a fixed pixel. Each window is then assigned an estimate and a confidence value. The value of the fixed pixel then takes the value of the estimate of the window with the highest confidence. This filter is not characterized by the same ambiguity in the presence of noise and manages to eliminate the block artifacts. The "Mean of Least Variance"(MLV) filter, proposed by M.A. Schulze also produces edge-enhancing smoothing results in images. Similarly to the Kuwahara filter it assumes a window of size 2 d − 1 × 2 d − 1 {\displaystyle 2d-1\times 2d-1} but instead of searching amongst four subregions of size d × d {\displaystyle d\times d} for the one with minimum variance it searches amongst all possible d × d {\displaystyle d\times d} subregions. This means the central pixel of the window will be assigned the mean of the one subregion out of a poss
Wide-column store
A wide-column store (or extensible record store) is a type of NoSQL database. It uses tables, rows, and columns, but unlike a relational database, the names and format of the columns can vary from row to row in the same table. A wide-column store can be interpreted as a two-dimensional key–value store. Google's Bigtable is one of the prototypical examples of a wide-column store. == Wide-column stores versus columnar databases == Wide-column stores such as Bigtable and Apache Cassandra are not column stores in the original sense of the term, since their two-level structures do not use a columnar data layout. In genuine column stores, a columnar data layout is adopted such that each column is stored separately on disk. Wide-column stores do often support the notion of column families that are stored separately. However, each such column family typically contains multiple columns that are used together, similar to traditional relational database tables. Within a given column family, all data is stored in a row-by-row fashion, such that the columns for a given row are stored together, rather than each column being stored separately. Wide-column stores that support column families are also known as column family databases. == Notable examples == Notable wide-column stores include: Apache Accumulo Apache Cassandra Apache HBase Bigtable DataStax Enterprise (uses Apache Cassandra) DataStax Astra DB (uses Apache Cassandra) Hypertable Azure Tables ScyllaDB
Stable Diffusion
Stable Diffusion is a deep learning, text-to-image model released in 2022 based on diffusion techniques. The generative artificial intelligence technology is the premier product of Stability AI and is considered to be a part of the ongoing AI boom. It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt. Its development involved researchers from the CompVis Group at LMU Munich and Runway with a computational donation from Stability and training data from non-profit organizations. Stable Diffusion is a latent diffusion model, a kind of deep generative artificial neural network. Its code and model weights have been released publicly, and an optimized version can run on most consumer hardware equipped with a modest GPU with as little as 2.4 GB VRAM. This marked a departure from previous proprietary text-to-image models such as DALL-E and Midjourney which were accessible only via cloud services. == Development == Stable Diffusion originated from a project called Latent Diffusion, developed in Germany by researchers at LMU Munich in Munich and Heidelberg University. Four of the original 5 authors (Robin Rombach, Andreas Blattmann, Patrick Esser and Dominik Lorenz) later joined Stability AI and released subsequent versions of Stable Diffusion. The technical license for the model was released by the CompVis group at LMU Munich. Development was led by Patrick Esser of Runway and Robin Rombach of CompVis, who were among the researchers who had earlier invented the latent diffusion model architecture used by Stable Diffusion. Stability AI also credited EleutherAI and LAION (a German nonprofit which assembled the dataset on which Stable Diffusion was trained) as supporters of the project. == Technology == === Architecture === Diffusion models, introduced in 2015, are trained with the objective of removing successive applications of Gaussian noise on training images, which can be thought of as a sequence of denoising autoencoders. The name diffusion is from the thermodynamic diffusion, since they were first developed with inspiration from thermodynamics. Models in Stable Diffusion series before SD 3 all used a variant of diffusion models, called latent diffusion model (LDM), developed in 2021 by the CompVis (Computer Vision & Learning) group at LMU Munich. Stable Diffusion consists of 3 parts: the variational autoencoder (VAE), U-Net, and an optional text encoder. The VAE encoder compresses the image from pixel space to a smaller dimensional latent space, capturing a more fundamental semantic meaning of the image. Gaussian noise is iteratively applied to the compressed latent representation during forward diffusion. The U-Net block, composed of a ResNet backbone, denoises the output from forward diffusion backwards to obtain a latent representation. Finally, the VAE decoder generates the final image by converting the representation back into pixel space. The denoising step can be flexibly conditioned on a string of text, an image, or another modality. The encoded conditioning data is exposed to denoising U-Nets via a cross-attention mechanism. For conditioning on text, the fixed, pretrained CLIP ViT-L/14 text encoder is used to transform text prompts to an embedding space. Researchers point to increased computational efficiency for training and generation as an advantage of LDMs. With 860 million parameters in the U-Net and 123 million in the text encoder, Stable Diffusion is considered relatively lightweight by 2022 standards, and unlike other diffusion models, it can run on consumer GPUs, and even CPU-only if using the OpenVINO version of Stable Diffusion. ==== SD XL ==== The XL version uses the same LDM architecture as previous versions, except larger: larger UNet backbone, larger cross-attention context, two text encoders instead of one, and trained on multiple aspect ratios (not just the square aspect ratio like previous versions). The SD XL Refiner, released at the same time, has the same architecture as SD XL, but it was trained for adding fine details to preexisting images via text-conditional img2img. ==== SD 3.0 ==== The 3.0 version completely changes the backbone. Not a UNet, but a Rectified Flow Transformer, which implements the rectified flow method with a Transformer. The Transformer architecture used for SD 3.0 has three "tracks", for original text encoding, transformed text encoding, and image encoding (in latent space). The transformed text encoding and image encoding are mixed during each transformer block. The architecture is named "multimodal diffusion transformer (MMDiT), where the "multimodal" means that it mixes text and image encodings inside its operations. This differs from previous versions of DiT, where the text encoding affects the image encoding, but not vice versa. === Training data === Stable Diffusion was trained on pairs of images and captions taken from LAION-5B, a publicly available dataset derived from Common Crawl data scraped from the web, where 5 billion image-text pairs were classified based on language and filtered into separate datasets by resolution, a predicted likelihood of containing a watermark, and predicted "aesthetic" score (e.g. subjective visual quality). The dataset was created by LAION, a German non-profit which receives funding from Stability AI. The Stable Diffusion model was trained on three subsets of LAION-5B: laion2B-en, laion-high-resolution, and laion-aesthetics v2 5+. A third-party analysis of the model's training data identified that out of a smaller subset of 12 million images taken from the original wider dataset used, approximately 47% of the sample size of images came from 100 different domains, with Pinterest taking up 8.5% of the subset, followed by websites such as WordPress, Blogspot, Flickr, DeviantArt and Wikimedia Commons. An investigation by Bayerischer Rundfunk showed that LAION's datasets, hosted on Hugging Face, contain large amounts of private and sensitive data. === Training procedures === The model was initially trained on the laion2B-en and laion-high-resolution subsets, with the last few rounds of training done on LAION-Aesthetics v2 5+, a subset of 600 million captioned images which the LAION-Aesthetics Predictor V2 predicted that humans would, on average, give a score of at least 5 out of 10 when asked to rate how much they liked them. The LAION-Aesthetics v2 5+ subset also excluded low-resolution images and images which LAION-5B-WatermarkDetection identified as carrying a watermark with greater than 80% probability. Final rounds of training additionally dropped 10% of text conditioning to improve Classifier-Free Diffusion Guidance. The model was trained using 256 Nvidia A100 GPUs on Amazon Web Services for a total of 150,000 GPU-hours, at a cost of $600,000. === Limitations === Stable Diffusion has issues with degradation and inaccuracies in certain scenarios. Initial releases of the model were trained on a dataset that consists of 512×512 resolution images, meaning that the quality of generated images noticeably degrades when user specifications deviate from its "expected" 512×512 resolution; the version 2.0 update of the Stable Diffusion model later introduced the ability to natively generate images at 768×768 resolution. Another challenge is in generating human limbs due to poor data quality of limbs in the LAION database. The model is insufficiently trained to replicate human limbs and faces due to the lack of representative features in the database, and prompting the model to generate images of such type can confound the model. In addition to human limbs, Stable Diffusion is unable to generate legible ambigrams and some other forms of text and typography. Stable Diffusion XL (SDXL) version 1.0, released in July 2023, introduced native 1024x1024 resolution and improved generation for limbs and text. Accessibility for individual developers can also be a problem. In order to customize the model for new use cases that are not included in the dataset, such as generating anime characters ("waifu diffusion"), new data and further training are required. Fine-tuned adaptations of Stable Diffusion created through additional retraining have been used for a variety of different use-cases, from medical imaging to algorithmically generated music. However, this fine-tuning process is sensitive to the quality of new data; low resolution images or different resolutions from the original data can not only fail to learn the new task but degrade the overall performance of the model. Even when the model is additionally trained on high quality images, it is difficult for individuals to run models in consumer electronics. For example, the training process for waifu-diffusion requires a minimum 30 GB of VRAM, which exceeds the usual resource provided in such consumer GPUs as Nvidia's GeForce 30 series, w
Hindsight optimization
Hindsight optimisation (HOP) is a computer science technique used in artificial intelligence for analysis of actions which have stochastic results. HOP is used in combination with a deterministic planner. By creating sample results for each of the possible actions from the given state (i.e. determinising the actions), and using the deterministic planner to analyse those sample results, HOP allows an estimate of the actual action.
Hindsight optimization
Hindsight optimisation (HOP) is a computer science technique used in artificial intelligence for analysis of actions which have stochastic results. HOP is used in combination with a deterministic planner. By creating sample results for each of the possible actions from the given state (i.e. determinising the actions), and using the deterministic planner to analyse those sample results, HOP allows an estimate of the actual action.
Quantum robotics
Quantum robotics is an interdisciplinary field that investigates the intersection of robotics and quantum mechanics. This field, in particular, explores the applications of quantum phenomena such as quantum entanglement within the realm of robotics. Examples of its applications include quantum communication in multi-agent cooperative robotic scenarios, the use of quantum algorithms in performing robotics tasks, and the integration of quantum devices (e.g., quantum detectors) in robotic systems. == Introduction == The free-space quantum communication between mobile platforms was proposed for reconfigurable quantum key distribution (QKD) applications using unmanned aerial vehicle (UAVs, a.k.a. drones) in 2017. This technology was later advanced in various aspects in mobile drone and vehicle platforms in several configurations such as drone-to-drone, drone-to-moving vehicle, and vehicle-to-vehicle systems. Some research has contributed to low-size, low-weight, and low-power quantum key distribution systems for small-form UAVs, the characterization of a polarization-based receiver for mobile free-space optical QKD, and optical-relayed entanglement distribution using drones as mobile nodes. The topic of free-space quantum communication between mobile platforms, initially developed to meet the need for free-space QKD and entanglement distribution using mobile nodes, was brought into the robotics domain as an emerging interdisciplinary mechatronics topic to investigate the interface between quantum technologies and the robotic systems domain. The main advantage of such integrated technology is the guaranteed security in communication between multi-agent and cooperative autonomous systems. Other advances are anticipated. == Quantum entanglement == According to quantum mechanics, entanglement occurs when more than one particle become connected. If the state of one particle changes then it will instantly change the state of other particles regardless of their distance. Entangled sensors do the same kind of work and achieve strong sensitivity. A group of quantum robots can measure magnetic fields, gravitational fields and other physical properties using entangled sensors with high rate of accuracy. Again the connection of one robot to other is increased (become strong) by quantum entanglement. == Quantum teleportation == Quantum teleportation is the transfer of quantum information (not physical objects). This is used in case of multi robot process. One robot is programmed with a complex quantum update. Then that robot can teleport that complex quantum information (the update) to other robots. This teleportation or communication is very secure because all the work is done in quantum state. == Kinematics == Quantum computing has been proposed as being optimal for calculating inverse kinematics values. == Alice and Bob robots == In the realm of quantum mechanics, the names Alice and Bob are frequently employed to illustrate various phenomena, protocols, and applications. These include their roles in QKD, quantum cryptography, entanglement, and teleportation. The terms "Alice Robot" and "Bob Robot" serve as analogous expressions that merge the concepts of Alice and Bob from quantum mechanics with mechatronic mobile platforms (such as robots, drones, and autonomous vehicles). For example, the Alice Robot functions as a transmitter platform that communicates with the Bob Robot, housing the receiving detectors.
Ensemble averaging (machine learning)
In machine learning, ensemble averaging is the process of creating multiple models (typically artificial neural networks) and combining them to produce a desired output, as opposed to creating just one model. Ensembles of models often outperform individual models, as the various errors of the ensemble constituents "average out". == Overview == Ensemble averaging is one of the simplest types of committee machines. Along with boosting, it is one of the two major types of static committee machines. In contrast to standard neural network design, in which many networks are generated but only one is kept, ensemble averaging keeps the less satisfactory networks, but with less weight assigned to their outputs. The theory of ensemble averaging relies on two properties of artificial neural networks: In any network, the bias can be reduced at the cost of increased variance In a group of networks, the variance can be reduced at no cost to the bias. This is known as the bias–variance tradeoff. Ensemble averaging creates a group of networks, each with low bias and high variance, and combines them to form a new network which should theoretically exhibit low bias and low variance. Hence, this can be thought of as a resolution of the bias–variance tradeoff. The idea of combining experts can be traced back to Pierre-Simon Laplace. == Method == The theory mentioned above gives an obvious strategy: create a set of experts with low bias and high variance, and average them. Generally, what this means is to create a set of experts with varying parameters; frequently, these are the initial synaptic weights of a neural network, although other factors (such as learning rate, momentum, etc.) may also be varied. Some authors recommend against varying weight decay and early stopping. The steps are therefore: Generate N experts, each with their own initial parameters (these values are usually sampled randomly from a distribution) Train each expert separately Combine the experts and average their values. Alternatively, domain knowledge may be used to generate several classes of experts. An expert from each class is trained, and then combined. A more complex version of ensemble average views the final result not as a mere average of all the experts, but rather as a weighted sum. If each expert is y i {\displaystyle y_{i}} , then the overall result y ~ {\displaystyle {\tilde {y}}} can be defined as: y ~ ( x ; α ) = ∑ j = 1 p α j y j ( x ) {\displaystyle {\tilde {y}}(\mathbf {x} ;\mathbf {\alpha } )=\sum _{j=1}^{p}\alpha _{j}y_{j}(\mathbf {x} )} where α {\displaystyle \mathbf {\alpha } } is a set of weights. The optimization problem of finding alpha is readily solved through neural networks, hence a "meta-network" where each "neuron" is in fact an entire neural network can be trained, and the synaptic weights of the final network is the weight applied to each expert. This is known as a linear combination of experts. It can be seen that most forms of neural network are some subset of a linear combination: the standard neural net (where only one expert is used) is simply a linear combination with all α j = 0 {\displaystyle \alpha _{j}=0} and one α k = 1 {\displaystyle \alpha _{k}=1} . A raw average is where all α j {\displaystyle \alpha _{j}} are equal to some constant value, namely one over the total number of experts. A more recent ensemble averaging method is negative correlation learning, proposed by Y. Liu and X. Yao. This method has been widely used in evolutionary computing. == Benefits == The resulting committee is almost always less complex than a single network that would achieve the same level of performance The resulting committee can be trained more easily on smaller datasets The resulting committee often has improved performance over any single model The risk of overfitting is lessened, as there are fewer parameters (e.g. neural network weights) which need to be set.