Marcus Hutter (born 14 April 1967 in Munich) is a German computer scientist, professor and artificial intelligence researcher. As a senior researcher at DeepMind, he studies the mathematical foundations of artificial general intelligence. Hutter studied physics and computer science at the Technical University of Munich. In 2000, he joined Jürgen Schmidhuber's group at the Dalle Molle Institute for Artificial Intelligence Research in Manno, Switzerland. He developed a mathematical formalism of artificial general intelligence named AIXI. He has served as a professor at the College of Engineering, Computing and Cybernetics of the Australian National University in Canberra, Australia. == Research == Starting in 2000, Hutter developed and published a mathematical theory of artificial general intelligence, AIXI, based on idealised intelligent agents and reward-motivated reinforcement learning. His first book Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability was published in 2005 by Springer. Also in 2005, Hutter published with his doctoral student Shane Legg an intelligence test for artificial intelligence devices. In 2009, Hutter developed and published the theory of feature reinforcement learning. In 2014, Lattimore and Hutter published an asymptotically optimal extension of the AIXI agent. An accessible podcast with Lex Fridman about his theory of Universal AI appeared in 2021 and a more technical follow-up with Tim Nguyen in 2024 in the Cartesian Cafe. His new (2024) book also gives a more accessible introduction to Universal AI and progress in the 20 years since his first book, including a chapter on ASI safety, which featured as a keynote at the inaugural workshop on AI safety in Sydney. == Hutter Prize == In 2006, Hutter announced the Hutter Prize for Lossless Compression of Human Knowledge, with a total of €50,000 in prize money. In 2020, Hutter raised the prize money for the Hutter Prize to €500,000.
Tom's Planner
Tom's Planner is a web-based tool and application service provider for project planning, management and collaboration. == History == Tom's Planner is based on Curaçao. In November 2009, it announced its public beta launch on TechCrunch and moved out of beta in August 2010. In 2013 Tom's Planner acquired its competitor Gantto. == Software == Tom's Planner is project management software that enables the creation of project schedules (Gantt charts) using a visual perspective. Tom's Planner uses the Freemium Business Model. Users can register for a free account or choose a paid version. Tom's Planner is available in five languages and is used by thousands of users on a daily basis in more than 100 countries worldwide. Customers range from fortune 500 companies to small mom-and-pop shops. == Reviews == Tom's Planner has been reviewed by PC World, TechCrunch, Lifehacker, and several other periodicals.
Round-trip translation
Round-trip translation (RTT), also known as back-and-forth translation, recursive translation and bi-directional translation, is the process of translating a word, phrase or text into another language (forward translation), then translating the result back into the original language (back translation), using machine translation (MT) software. It is often used by laypeople to evaluate a machine translation system, or to test whether a text is suitable for MT when they are unfamiliar with the target language. Because the resulting text can often differ substantially from the original, RTT can also be a source of entertainment. == Software quality == To compare the quality of different machine translation systems, users perform RTT and compare the resulting text to the original. The theory is that the closer the result of the RTT is to the original text, the higher the quality of the machine translation system. One of the problems with this technique is that if there is a problem with the resulting text it is impossible to know whether the error occurred in the forward translation, in the back translation, or in both. In addition, it is possible to get a good back translation from a bad forward translation. A study using the automatic evaluation methods BLEU and F-score compared five different free online translation programs, evaluating the quality of both the forward translation and the back translation, and found no correlation between the quality of the forward translation and the quality of the back translation (i.e., a high quality forward translation did not always correspond to a high quality back translation). The author concluded that RTT was a poor method of predicting the quality of machine translation software. This conclusion was reinforced by a more in-depth study also using automatic evaluation methods. A subsequent study which included human evaluation of the back translation in addition to automatic evaluation methods found that RTT might have some ability to predict the quality of a machine translation system not on a sentence-by-sentence basis but for larger texts. == Suitability of text for machine translation == It is also suggested that RTT can be used to determine whether a text is suitable for machine translation. The idea being that if RTT results in a text that is close to the original, the text is suitable for MT. If after using RTT, the resulting text is inaccurate, the source text can then be edited until a satisfactory result is achieved. One of the studies looking at RTT as a means of measuring MT system quality also looked at its ability to predict whether a text was suitable for machine translation. It found that using different types of text also did not result in any correlation between the quality of the forward translation and the quality of the back translation. In contrast another study using human evaluation found that there was a correlation between the quality of the forward translation and the back translation and that this correlation could be used to estimate the quality of the forward translation. This correlation could be used to estimate the quality of the forward translation and by simplifying the source text, improve the quality of the forward translation. == Entertainment == Although the use of RTT for assessing MT system quality or the suitability of a text for MT is in doubt, it is a way to have fun with machine translation. The text produced from an RTT can be comically bad. At one time websites existed for the sole purpose of performing RTT for fun. Other variations send the text through several languages before translating it back into the original or continue translating the text back and forth until it reaches equilibrium (i.e., the result of the back translation is identical to the text used for the forward translation). RTT as entertainment appeared in Philip K. Dick's novel Galactic Pot-Healer. The main character runs book titles and sayings through RTT then has his friends try to guess the original. The Australian television show Spicks and Specks had a contest called "Turning Japanese" which used RTT on song lyrics. Contestants needed to correctly guess the title of the song from which the lyrics were taken.
Tamara Broderick
Tamara Ann Broderick is an American computer scientist at the Massachusetts Institute of Technology. She works on machine learning and Bayesian inference. == Education and early career == Broderick is from Parma Heights, Ohio. She attended Laurel School and graduated in 2003. Whilst at high school she took part in the inaugural Massachusetts Institute of Technology Women's Technology Program. She studied mathematics at Princeton University, earning a bachelor's degree in 2007. She was a Marshall scholar, allowing her to pursue graduate research at the University of Cambridge. She was a runner-up in the Association for Women in Mathematics Alice T. Shafer Prize for Excellence in Mathematics. She was co-president of the Princeton Math Club and organised a competition for high school maths teams. She won the Phi Beta Kappa Prize for the highest academic average at Princeton University. During her undergraduate degree, Broderick worked on dark matter haloes with Rachel Mandelbaum. Broderick moved to the United Kingdom for her graduate studies, earning a Master of Advanced Studies for completing Part III of the Mathematical Tripos at the University of Cambridge in 2009. Her Master's thesis looked at the Nomon selection method, improving the efficiency of communications. She returned to America in 2009, joining University of California, Berkeley for her Master's and PhD. Her graduate research was supported by the Berkeley Fellowship and a National Science Foundation Fellowship. Her PhD thesis Clusters and features from combinatorial stochastic processes looked at clustering and speeding up the analysis of large, streaming data sets. In 2013 she was selected for the Berkeley EECS Rising Stars conference. == Research and career == Broderick joined Massachusetts Institute of Technology as an assistant professor in 2015. She is interested in Bayesian statistics and graphical models. She was the recipient of a Google Faculty Research Grant and International Society for Bayesian Analysis Lifetime Members Junior Researcher Award. She was awarded an Army Research Office young investigator program award to investigate machine-learning to quantify uncertainty in data analysis. Broderick is also Alfred P. Sloan Foundation scholar. === Academic service === In 2018, Broderick spoke at the Harvard University Institute for Applied Computational Science Women in Data Science conference. She spoke about Bayesian inference at the 2018 International Conference on Machine Learning. She led a three-day Masterclass on machine learning at University College London in June 2018. Broderick is a scientific advisor for AI.Reverie and WiML (Women in Machine Learning). She has developed a high-school level introduction to machine learning with the Women's Technology Program (WTP). Software she has developed is available on her website. === Awards and honors === Broderick was awarded the Evelyn Fix Memorial Medal and Citation and the International Society for Bayesian Analysis Savage Award for her doctoral thesis. She was awarded a National Science Foundation CAREER Award to scale her machine learning techniques. She was a 2021 Leadership Academy winner of the Committee of Presidents of Statistical Societies.
Brian D. Ripley
Brian David Ripley FRSE (born 29 April 1952) is a British statistician. From 1990, he was professor of applied statistics at the University of Oxford and also a professorial fellow at St Peter's College. He retired August 2014 due to ill health. == Biography == Ripley has made contributions to the fields of spatial statistics and pattern recognition. His work on artificial neural networks in the 1990s helped to bring aspects of machine learning and data mining to the attention of statistical audiences. He emphasised the value of robust statistics in his books Pattern Recognition and Neural Networks and Modern Applied Statistics with S. Ripley helped develop the S-PLUS programming language and its open source derivative R. He co-authored two books based on S, S Programming and Modern Applied Statistics with S. Since mid-1997 he is a member of the "R Core Team" and from 2000 to 2021 he was one of the most active committers to the R core. The package MASS is one of only fifteen "recommended packages" for R (with June 2024 more than 20,900). He was educated at the University of Cambridge, where he was awarded both the Smith's Prize (at the time awarded to the best graduate essay writer who had been undergraduate at Cambridge in that cohort) and the Rollo Davidson Prize. The university also awarded him the Adams Prize in 1987 for an essay entitled Statistical Inference for Spatial Processes, later published as a book. He served on the faculty of Imperial College, London from 1976 until 1983, at which point he moved to the University of Strathclyde. == Authored books == Ripley, B. D. (1981) Spatial Statistics. Wiley, 252pp. ISBN 0-471-08367-4. Ripley, B. D. (1983) Stochastic Simulation. Wiley, ISBN 0-471-81884-4. Ripley, B. D. (1988). Statistical Inference for Spatial Processes. Cambridge University Press. ISBN 0-521-35234-7. Ripley, B. D. (1996) Pattern Recognition and Neural Networks. Cambridge University Press. 403 pages. ISBN 0-521-46086-7. Venables, W. N. and Ripley, B. D. (2000) S Programming. Springer, 264pp. ISBN 978-0-387-98966-2. Venables, W. N. and Ripley, B. D. (2002) Modern Applied Statistics with S (Fourth Edition; previous editions published as Modern Applied Statistics with S-PLUS in 1994, 1997 & 1999). Springer, 462pp. ISBN 978-0-387-95457-8.
Contrastive Language-Image Pre-training
Contrastive Language-Image Pre-training (CLIP) is a technique for training a pair of neural network models, one for image understanding and one for text understanding, using a contrastive objective. This method has enabled broad applications across multiple domains, including cross-modal retrieval, text-to-image generation, and aesthetic ranking. == Algorithm == The CLIP method trains a pair of models contrastively. One model takes in a piece of text as input and outputs a single vector representing its semantic content. The other model takes in an image and similarly outputs a single vector representing its visual content. The models are trained so that the vectors corresponding to semantically similar text-image pairs are close together in the shared vector space, while those corresponding to dissimilar pairs are far apart. To train a pair of CLIP models, one would start by preparing a large dataset of image-caption pairs. During training, the models are presented with batches of N {\displaystyle N} image-caption pairs. Let the outputs from the text and image models be respectively v 1 , . . . , v N , w 1 , . . . , w N {\displaystyle v_{1},...,v_{N},w_{1},...,w_{N}} . Two vectors are considered "similar" if their dot product is large. The loss incurred on this batch is the multi-class N-pair loss, which is a symmetric cross-entropy loss over similarity scores: − 1 N ∑ i ln e v i ⋅ w i / T ∑ j e v i ⋅ w j / T − 1 N ∑ j ln e v j ⋅ w j / T ∑ i e v i ⋅ w j / T {\displaystyle -{\frac {1}{N}}\sum _{i}\ln {\frac {e^{v_{i}\cdot w_{i}/T}}{\sum _{j}e^{v_{i}\cdot w_{j}/T}}}-{\frac {1}{N}}\sum _{j}\ln {\frac {e^{v_{j}\cdot w_{j}/T}}{\sum _{i}e^{v_{i}\cdot w_{j}/T}}}} In essence, this loss function encourages the dot product between matching image and text vectors ( v i ⋅ w i {\displaystyle v_{i}\cdot w_{i}} ) to be high, while discouraging high dot products between non-matching pairs. The parameter T > 0 {\displaystyle T>0} is the temperature, which is parameterized in the original CLIP model as T = e − τ {\displaystyle T=e^{-\tau }} where τ ∈ R {\displaystyle \tau \in \mathbb {R} } is a learned parameter. Other loss functions are possible. For example, Sigmoid CLIP (SigLIP) proposes the following loss function: L = 1 N ∑ i , j ∈ 1 : N f ( ( 2 δ i , j − 1 ) ( e τ w i ⋅ v j + b ) ) {\displaystyle L={\frac {1}{N}}\sum _{i,j\in 1:N}f((2\delta _{i,j}-1)(e^{\tau }w_{i}\cdot v_{j}+b))} where f ( x ) = ln ( 1 + e − x ) {\displaystyle f(x)=\ln(1+e^{-x})} is the negative log sigmoid loss, and the Dirac delta symbol δ i , j {\displaystyle \delta _{i,j}} is 1 if i = j {\displaystyle i=j} else 0. == CLIP models == While the original model was developed by OpenAI, subsequent models have been trained by other organizations as well. === Image model === The image encoding models used in CLIP are typically vision transformers (ViT). The naming convention for these models often reflects the specific ViT architecture used. For instance, "ViT-L/14" means a "vision transformer large" (compared to other models in the same series) with a patch size of 14, meaning that the image is divided into 14-by-14 pixel patches before being processed by the transformer. The size indicator ranges from B, L, H, G (base, large, huge, giant), in that order. Other than ViT, the image model is typically a convolutional neural network, such as ResNet (in the original series by OpenAI), or ConvNeXt (in the OpenCLIP model series by LAION). Since the output vectors of the image model and the text model must have exactly the same length, both the image model and the text model have fixed-length vector outputs, which in the original report is called "embedding dimension". For example, in the original OpenAI model, the ResNet models have embedding dimensions ranging from 512 to 1024, and for the ViTs, from 512 to 768. Its implementation of ViT was the same as the original one, with one modification: after position embeddings are added to the initial patch embeddings, there is a LayerNorm. Its implementation of ResNet was the same as the original one, with 3 modifications: In the start of the CNN (the "stem"), they used three stacked 3x3 convolutions instead of a single 7x7 convolution, as suggested by. There is an average pooling of stride 2 at the start of each downsampling convolutional layer (they called it rect-2 blur pooling according to the terminology of ). This has the effect of blurring images before downsampling, for antialiasing. The final convolutional layer is followed by a multiheaded attention pooling. ALIGN a model with similar capabilities, trained by researchers from Google used EfficientNet, a kind of convolutional neural network. === Text model === The text encoding models used in CLIP are typically Transformers. In the original OpenAI report, they reported using a Transformer (63M-parameter, 12-layer, 512-wide, 8 attention heads) with lower-cased byte pair encoding (BPE) with 49152 vocabulary size. Context length was capped at 76 for efficiency. Like GPT, it was decoder-only, with only causally-masked self-attention. Its architecture is the same as GPT-2. Like BERT, the text sequence is bracketed by two special tokens [SOS] and [EOS] ("start of sequence" and "end of sequence"). Take the activations of the highest layer of the transformer on the [EOS], apply LayerNorm, then a final linear map. This is the text encoding of the input sequence. The final linear map has output dimension equal to the embedding dimension of whatever image encoder it is paired with. These models all had context length 77 and vocabulary size 49408. ALIGN used BERT of various sizes. == Dataset == === WebImageText === The CLIP models released by OpenAI were trained on a dataset called "WebImageText" (WIT) containing 400 million pairs of images and their corresponding captions scraped from the internet. The total number of words in this dataset is similar in scale to the WebText dataset used for training GPT-2, which contains about 40 gigabytes of text data. The dataset contains 500,000 text-queries, with up to 20,000 (image, text) pairs per query. The text-queries were generated by starting with all words occurring at least 100 times in English Wikipedia, then extended by bigrams with high mutual information, names of all Wikipedia articles above a certain search volume, and WordNet synsets. The dataset is private and has not been released to the public, and there is no further information on it. ==== Data preprocessing ==== For the CLIP image models, the input images are preprocessed by first dividing each of the R, G, B values of an image by the maximum possible value, so that these values fall between 0 and 1, then subtracting by [0.48145466, 0.4578275, 0.40821073], and dividing by [0.26862954, 0.26130258, 0.27577711]. The rationale was that these are the mean and standard deviations of the images in the WebImageText dataset, so this preprocessing step roughly whitens the image tensor. These numbers slightly differ from the standard preprocessing for ImageNet, which uses [0.485, 0.456, 0.406] and [0.229, 0.224, 0.225]. If the input image does not have the same resolution as the native resolution (224×224 for all except ViT-L/14@336px, which has 336×336 resolution), then the input image is first scaled by bicubic interpolation, so that its shorter side is the same as the native resolution, then the central square of the image is cropped out. === Others === ALIGN used over one billion image-text pairs, obtained by extracting images and their alt-tags from online crawling. The method was described as similar to how the Conceptual Captions dataset was constructed, but instead of complex filtering, they only applied a frequency-based filtering. Later models trained by other organizations had published datasets. For example, LAION trained OpenCLIP with published datasets LAION-400M, LAION-2B, and DataComp-1B. == Training == In the original OpenAI CLIP report, they reported training 5 ResNet and 3 ViT (ViT-B/32, ViT-B/16, ViT-L/14). Each was trained for 32 epochs. The largest ResNet model took 18 days to train on 592 V100 GPUs. The largest ViT model took 12 days on 256 V100 GPUs. All ViT models were trained on 224×224 image resolution. The ViT-L/14 was then boosted to 336×336 resolution by FixRes, resulting in a model. They found this was the best-performing model. In the OpenCLIP series, the ViT-L/14 model was trained on 384 A100 GPUs on the LAION-2B dataset, for 160 epochs for a total of 32B samples seen. == Applications == === Cross-modal retrieval === CLIP's cross-modal retrieval enables the alignment of visual and textual data in a shared latent space, allowing users to retrieve images based on text descriptions and vice versa, without the need for explicit image annotations. In text-to-image retrieval, users input descriptive text, and CLIP retrieves images with matching embeddings. In image-to-text retrieval, images are used to find related text content. CLIP’s ability to connect vis
JOONE
JOONE (Java Object Oriented Neural Engine) is a component based neural network framework built in Java. == Features == Joone consists of a component-based architecture based on linkable components that can be extended to build new learning algorithms and neural networks architectures. Components are plug-in code modules that are linked to produce an information flow. New components can be added and reused. Beyond simulation, Joone also has to some extent multi-platform deployment capabilities. Joone has a GUI Editor to graphically create and test any neural network, and a distributed training environment that allows for neural networks to be trained on multiple remote machines. == Comparison == As of 2010, Joone, Encog and Neuroph are the major free component based neural network development environment available for the Java platform. Unlike the two other (commercial) systems that are in existence, Synapse and NeuroSolutions, it is written in Java and has direct cross-platform support. A limited number of components exist and the graphical development environment is rudimentary so it has significantly fewer features than its commercial counterparts. Joone can be considered to be more of a neural network framework than a full integrated development environment. Unlike its commercial counterparts, it has a strong focus on code-based development of neural networks rather than visual construction. While in theory Joone can be used to construct a wider array of adaptive systems (including those with non-adaptive elements), its focus is on backpropagation based neural networks.