Language resource management – Lexical markup framework (LMF; ISO 24613), produced by ISO/TC 37, is the ISO standard for natural language processing (NLP) and machine-readable dictionary (MRD) lexicons. The scope is standardization of principles and methods relating to language resources in the contexts of multilingual communication. == Objectives == The goals of LMF are to provide a common model for the creation and use of lexical resources, to manage the exchange of data between and among these resources, and to enable the merging of large number of individual electronic resources to form extensive global electronic resources. Types of individual instantiations of LMF can include monolingual, bilingual or multilingual lexical resources. The same specifications are to be used for both small and large lexicons, for both simple and complex lexicons, for both written and spoken lexical representations. The descriptions range from morphology, syntax, computational semantics to computer-assisted translation. The covered languages are not restricted to European languages but cover all natural languages. The range of targeted NLP applications is not restricted. LMF is able to represent most lexicons, including WordNet, EDR and PAROLE lexicons. == History == In the past, lexicon standardization has been studied and developed by a series of projects like GENELEX, EDR, EAGLES, MULTEXT, PAROLE, SIMPLE and ISLE. Then, the ISO/TC 37 National delegations decided to address standards dedicated to NLP and lexicon representation. The work on LMF started in Summer 2003 by a new work item proposal issued by the US delegation. In Fall 2003, the French delegation issued a technical proposition for a data model dedicated to NLP lexicons. In early 2004, the ISO/TC 37 committee decided to form a common ISO project with Nicoletta Calzolari (CNR-ILC Italy) as convenor and Gil Francopoulo (Tagmatica France) and Monte George (ANSI, United States) as editors. The first step in developing LMF was to design an overall framework based on the general features of existing lexicons and to develop a consistent terminology to describe the components of those lexicons. The next step was the actual design of a comprehensive model that best represented all of the lexicons in detail. A large panel of 60 experts contributed a wide range of requirements for LMF that covered many types of NLP lexicons. The editors of LMF worked closely with the panel of experts to identify the best solutions and reach a consensus on the design of LMF. Special attention was paid to the morphology in order to provide powerful mechanisms for handling problems in several languages that were known as difficult to handle. 13 versions have been written, dispatched (to the National nominated experts), commented and discussed during various ISO technical meetings. After five years of work, including numerous face-to-face meetings and e-mail exchanges, the editors arrived at a coherent UML model. In conclusion, LMF should be considered a synthesis of the state of the art in NLP lexicon field. == Current stage == The ISO number is 24613. The LMF specification has been published officially as an International Standard on 17 November 2008. == As one of the members of the ISO/TC 37 family of standards == The ISO/TC 37 standards are currently elaborated as high level specifications and deal with word segmentation (ISO 24614), annotations (ISO 24611 a.k.a. MAF, ISO 24612 a.k.a. LAF, ISO 24615 a.k.a. SynAF, and ISO 24617-1 a.k.a. SemAF/Time), feature structures (ISO 24610), multimedia containers (ISO 24616 a.k.a. MLIF), and lexicons (ISO 24613). These standards are based on low level specifications dedicated to constants, namely data categories (revision of ISO 12620), language codes (ISO 639), scripts codes (ISO 15924), country codes (ISO 3166) and Unicode (ISO 10646). The two level organization forms a coherent family of standards with the following common and simple rules: the high level specification provides structural elements that are adorned by the standardized constants; the low level specifications provide standardized constants as metadata. == Key standards == The linguistics constants like /feminine/ or /transitive/ are not defined within LMF but are recorded in the Data Category Registry (DCR) that is maintained as a global resource by ISO/TC 37 in compliance with ISO/IEC 11179-3:2003. And these constants are used to adorn the high level structural elements. The LMF specification complies with the modeling principles of Unified Modeling Language (UML) as defined by Object Management Group (OMG). The structure is specified by means of UML class diagrams. The examples are presented by means of UML instance (or object) diagrams. An XML DTD is given in an annex of the LMF document. == Model structure == LMF is composed of the following components: The core package that is the structural skeleton which describes the basic hierarchy of information in a lexical entry. Extensions of the core package which are expressed in a framework that describes the reuse of the core components in conjunction with the additional components required for a specific lexical resource. The extensions are specifically dedicated to morphology, MRD, NLP syntax, NLP semantics, NLP multilingual notations, NLP morphological patterns, multiword expression patterns, and constraint expression patterns. == Example == In the following example, the lexical entry is associated with a lemma clergyman and two inflected forms clergyman and clergymen. The language coding is set for the whole lexical resource. The language value is set for the whole lexicon as shown in the following UML instance diagram. The elements Lexical Resource, Global Information, Lexicon, Lexical Entry, Lemma, and Word Form define the structure of the lexicon. They are specified within the LMF document. On the contrary, languageCoding, language, partOfSpeech, commonNoun, writtenForm, grammaticalNumber, singular, plural are data categories that are taken from the Data Category Registry. These marks adorn the structure. The values ISO 639-3, clergyman, clergymen are plain character strings. The value eng is taken from the list of languages as defined by ISO 639-3. With some additional information like dtdVersion and feat, the same data can be expressed by the following XML fragment: This example is rather simple, while LMF can represent much more complex linguistic descriptions the XML tagging is correspondingly complex. == Selected publications about LMF == The first publication about the LMF specification as it has been ratified by ISO (this paper became (in 2015) the 9th most cited paper within the Language Resources and Evaluation conferences from LREC papers): Language Resources and Evaluation LREC-2006/Genoa: Gil Francopoulo, Monte George, Nicoletta Calzolari, Monica Monachini, Nuria Bel, Mandy Pet, Claudia Soria: Lexical Markup Framework (LMF) About semantic representation: Gesellschaft für linguistische Datenverarbeitung GLDV-2007/Tübingen: Gil Francopoulo, Nuria Bel, Monte George Nicoletta Calzolari, Monica Monachini, Mandy Pet, Claudia Soria: Lexical Markup Framework ISO standard for semantic information in NLP lexicons About African languages: Traitement Automatique des langues naturelles, Marseille, 2014: Mouhamadou Khoule, Mouhamad Ndiankho Thiam, El Hadj Mamadou Nguer: Toward the establishment of a LMF-based Wolof language lexicon (Vers la mise en place d'un lexique basé sur LMF pour la langue wolof) [in French] About Asian languages: Lexicography, Journal of ASIALEX, Springer 2014: Lexical Markup Framework: Gil Francopoulo, Chu-Ren Huang: An ISO Standard for Electronic Lexicons and its Implications for Asian Languages DOI 10.1007/s40607-014-0006-z About European languages: COLING 2010: Verena Henrich, Erhard Hinrichs: Standardizing Wordnets in the ISO Standard LMF: Wordnet-LMF for GermaNet EACL 2012: Judith Eckle-Kohler, Iryna Gurevych: Subcat-LMF: Fleshing out a standardized format for subcategorization frame interoperability EACL 2012: Iryna Gurevych, Judith Eckle-Kohler, Silvana Hartmann, Michael Matuschek, Christian M Meyer, Christian Wirth: UBY - A Large-Scale Unified Lexical-Semantic Resource Based on LMF. About Semitic languages: Journal of Natural Language Engineering, Cambridge University Press (to appear in Spring 2015): Aida Khemakhem, Bilel Gargouri, Abdelmajid Ben Hamadou, Gil Francopoulo: ISO Standard Modeling of a large Arabic Dictionary. Proceedings of the seventh Global Wordnet Conference 2014: Nadia B M Karmani, Hsan Soussou, Adel M Alimi: Building a standardized Wordnet in the ISO LMF for aeb language. Proceedings of the workshop: HLT & NLP within Arabic world, LREC 2008: Noureddine Loukil, Kais Haddar, Abdelmajid Ben Hamadou: Towards a syntactic lexicon of Arabic Verbs. Traitement Automatique des Langues Naturelles, Toulouse (in French) 2007: Khemakhem A, Gargouri B, Abdelwahed A, Francopoulo G: Modélisation des paradigmes de fl
Human-centered AI
Human-centered AI is the initiative at the intersection of the fields of artificial intelligence (AI) and human-computer interaction (HCI) to develop AI systems in a way that prioritizes human values, needs, and general flourishing. Emphasis is placed on the recognition that artificial intelligence systems are rapidly changing, and will continue to influence, many aspects of the human experience, in areas ranging from scientific inquiry, governance and policy, labor and the economy, and creative expression, with an aim set to adapt current developments and guide future developments on a trajectory which is most beneficial to the human population at large, with the goal of augmenting human intelligence and capacities across these areas, as opposed to replacing them. Particular attention is paid to mitigating negative effects of AI automation on the livelihoods of the labor force, the use of AI in healthcare fields, and imbuing AI systems with societal values. Human-centered AI is linked to related endeavors in AI alignment and AI safety, but while these fields primarily focus on mitigating risks posed by AI that is unaligned to human values and/or uncontrollable AI self-development, human-centered AI places significant focus in exploring how AI systems can augment human capacities and serve as collaborators. == Conceptual history == The importance of the alignment of artificial intelligence development towards human values in some sense predates artificial intelligence itself, as before the modern conception of artificial intelligence as coined at the 1956 Dartmouth Workshop, the conception of robots as constructed, autonomous agents entered the cultural consciousness as early as the 1920s, with Karel Capek's Rossum's Universal Robots. The imagined issues relating to robots' aims and values requiring intentional alignment and direction with those of humans followed soon after, most widely known from science fiction author Isaac Asimov’s Three Laws of Robotics, dating to his 1942 short story “Runaround”. Two of the three eponymous laws are directly concerned with robots’ interaction with and positioned deference towards humans, and have in recent times been reexamined in the face of modern AI. In 1985, after artificial intelligence research had taken off and its effects were more acutely conceptualized, Asimov added a Rule Zero, treating robots' relationship with humanity as a whole, distinct from individual humans. While modern artificial intelligence is largely distinct from robotics, the conceptualization of both robots and AI systems as autonomous agents positions this as a foundation for conceptions of human-centered AI. Aside from robots, artificially intelligent autonomous agents in interaction with humans have been conceived of for at least 75 years. In 1950, Alan Turing published his famous "Imitation Game", often also called the Turing Test, a thought experiment that uses human-machine interaction as an assessor for the intelligence of a system. In recent times, artificial intelligence researchers such as Stanford's Erik Brynjolfsson have conceived of rapid AI development leading to a so-called "Turing Trap". == Augmentation and automation == A major stated aim of human-centered AI is to promote the development of AI in ways that augment human capabilities, rather than replacing them. To this end, organizations and initiatives that take a human-centered approach to AI development focus on frameworks that encourage collaboration between humans and artificial intelligence systems to build towards even greater progress, rather than attempting to automate tasks currently handled by humans. Such avenues include everything from data visualization for big data, allowing human engineers to better understand extremely large datasets, allowing for the design of better machine learning models to handle them, to AI-powered sensors to monitor vitals, allowing for better responsiveness from healthcare providers. Many human-centered AI initiatives often position it as a better alternative to the apparent mainstream in AI development, which is primarily concerned with automation. Driven by the pressures of the market economy, AI development that does replace tasks currently performed by humans with automated processes is incentivized, as it allows for greater profit margins; this often comes at the detriment of the human whose performance is replaced, thus leading to an environment wherein human workers are outcompeted by AI systems across various service-sector and technology-based industries. At the same time, automation and augmentation are not always incompatible; a major aim of human-centered AI is towards the automation of rote tasks that would otherwise hinder a human’s productivity or creativity, freeing them to direct their energy and intelligence towards higher-level tasks, thus achieving augmentation through automation. Empirical research in pharmaceutical sales has shown that a human-centered implementation - where work procedures, training, and incentives are designed around individuals' cognitive needs - improves augmentation performance, while implementation without such adaptation can worsen outcomes relative to a legacy system. == Research == Much of the work done on human-centered AI comes from research institutes, within universities, companies, and as freestanding organizations. The Stanford Institute for Human-Centered AI (abbreviated to HAI) is one such group, engaging academics, industry professionals, and policymakers centered in Stanford University to conduct research and inform policy in various areas in human-centered AI, including on aspects of the intelligence itself, augmentation, and on measuring the impacts of AI systems on sociopolitcal and cultural institutions. Similar groups exist at other universities, including the Chicago Human + AI (CHAI) Lab at the University of Chicago, the HCAI@GU group at the University of Gothenburg, and the Human-Centered AI (HAI) Lab at the University of Oxford. Outside of the academy, companies such as IBM have research initiatives dedicated to advancements in human-centered AI. At Kenyon College, the Integrated Program for Humane Studies (IPHS) launched a human-centered AI program in 2016 integrating artificial intelligence research with humanities and social science inquiry. This approach treats computation and humanistic scholarship as a single unified field of research rather than as separate disciplines requiring collaboration. The program's researchers have published in both AI venues (such as the International Conference on Machine Learning and Frontiers of Computer Science) and humanities journals (such as PMLA and Poetics Today), and the lab was selected in December 2025 by Schmidt Sciences for its Humanities and AI Virtual Institute to apply AI methods to cultural heritage preservation.
Neural architecture search
Neural architecture search (NAS) is a technique for automating the design of artificial neural networks (ANN), a widely used model in the field of machine learning. NAS has been used to design networks that are on par with or outperform hand-designed architectures. Methods for NAS can be categorized according to the search space, search strategy and performance estimation strategy used: The search space defines the type(s) of ANN that can be designed and optimized. The search strategy defines the approach used to explore the search space. The performance estimation strategy evaluates the performance of a possible ANN from its design (without constructing and training it). NAS is closely related to hyperparameter optimization and meta-learning and is a subfield of automated machine learning (AutoML). == Reinforcement learning == Reinforcement learning (RL) can underpin a NAS search strategy. Barret Zoph and Quoc Viet Le applied NAS with RL targeting the CIFAR-10 dataset and achieved a network architecture that rivals the best manually-designed architecture for accuracy, with an error rate of 3.65, 0.09 percent better and 1.05x faster than a related hand-designed model. On the Penn Treebank dataset, that model composed a recurrent cell that outperforms LSTM, reaching a test set perplexity of 62.4, or 3.6 perplexity better than the prior leading system. On the PTB character language modeling task it achieved bits per character of 1.214. Learning a model architecture directly on a large dataset can be a lengthy process. NASNet addressed this issue by transferring a building block designed for a small dataset to a larger dataset. The design was constrained to use two types of convolutional cells to return feature maps that serve two main functions when convoluting an input feature map: normal cells that return maps of the same extent (height and width) and reduction cells in which the returned feature map height and width is reduced by a factor of two. For the reduction cell, the initial operation applied to the cell's inputs uses a stride of two (to reduce the height and width). The learned aspect of the design included elements such as which lower layer(s) each higher layer took as input, the transformations applied at that layer and to merge multiple outputs at each layer. In the studied example, the best convolutional layer (or "cell") was designed for the CIFAR-10 dataset and then applied to the ImageNet dataset by stacking copies of this cell, each with its own parameters. The approach yielded accuracy of 82.7% top-1 and 96.2% top-5. This exceeded the best human-invented architectures at a cost of 9 billion fewer FLOPS—a reduction of 28%. The system continued to exceed the manually-designed alternative at varying computation levels. The image features learned from image classification can be transferred to other computer vision problems. E.g., for object detection, the learned cells integrated with the Faster-RCNN framework improved performance by 4.0% on the COCO dataset. In the so-called Efficient Neural Architecture Search (ENAS), a controller discovers architectures by learning to search for an optimal subgraph within a large graph. The controller is trained with policy gradient to select a subgraph that maximizes the validation set's expected reward. The model corresponding to the subgraph is trained to minimize a canonical cross entropy loss. Multiple child models share parameters, ENAS requires fewer GPU-hours than other approaches and 1000-fold less than "standard" NAS. On CIFAR-10, the ENAS design achieved a test error of 2.89%, comparable to NASNet. On Penn Treebank, the ENAS design reached test perplexity of 55.8. == Evolution == An alternative approach to NAS is based on evolutionary algorithms, which has been employed by several groups. An Evolutionary Algorithm for Neural Architecture Search generally performs the following procedure. First a pool consisting of different candidate architectures along with their validation scores (fitness) is initialised. At each step the architectures in the candidate pool are mutated (e.g.: 3x3 convolution instead of a 5x5 convolution). Next the new architectures are trained from scratch for a few epochs and their validation scores are obtained. This is followed by replacing the lowest scoring architectures in the candidate pool with the better, newer architectures. This procedure is repeated multiple times and thus the candidate pool is refined over time. Mutations in the context of evolving ANNs are operations such as adding or removing a layer, which include changing the type of a layer (e.g., from convolution to pooling), changing the hyperparameters of a layer, or changing the training hyperparameters. On CIFAR-10 and ImageNet, evolution and RL performed comparably, while both slightly outperformed random search. == Bayesian optimization == Bayesian Optimization (BO), which has proven to be an efficient method for hyperparameter optimization, can also be applied to NAS. In this context, the objective function maps an architecture to its validation error after being trained for a number of epochs. At each iteration, BO uses a surrogate to model this objective function based on previously obtained architectures and their validation errors. One then chooses the next architecture to evaluate by maximizing an acquisition function, such as expected improvement, which provides a balance between exploration and exploitation. Acquisition function maximization and objective function evaluation are often computationally expensive for NAS, and make the application of BO challenging in this context. Recently, BANANAS has achieved promising results in this direction by introducing a high-performing instantiation of BO coupled to a neural predictor. == Hill-climbing == Another group used a hill climbing procedure that applies network morphisms, followed by short cosine-annealing optimization runs. The approach yielded competitive results, requiring resources on the same order of magnitude as training a single network. E.g., on CIFAR-10, the method designed and trained a network with an error rate below 5% in 12 hours on a single GPU. == Multi-objective search == While most approaches solely focus on finding architecture with maximal predictive performance, for most practical applications other objectives are relevant, such as memory consumption, model size or inference time (i.e., the time required to obtain a prediction). Because of that, researchers created a multi-objective search. LEMONADE is an evolutionary algorithm that adopted Lamarckism to efficiently optimize multiple objectives. In every generation, child networks are generated to improve the Pareto frontier with respect to the current population of ANNs. Neural Architect is claimed to be a resource-aware multi-objective RL-based NAS with network embedding and performance prediction. Network embedding encodes an existing network to a trainable embedding vector. Based on the embedding, a controller network generates transformations of the target network. A multi-objective reward function considers network accuracy, computational resource and training time. The reward is predicted by multiple performance simulation networks that are pre-trained or co-trained with the controller network. The controller network is trained via policy gradient. Following a modification, the resulting candidate network is evaluated by both an accuracy network and a training time network. The results are combined by a reward engine that passes its output back to the controller network. == One-shot models == RL or evolution-based NAS require thousands of GPU-days of searching/training to achieve state-of-the-art computer vision results as described in the NASNet, mNASNet and MobileNetV3 papers. To reduce computational cost, many recent NAS methods rely on the weight-sharing idea. In this approach, a single overparameterized supernetwork (also known as the one-shot model) is defined. A supernetwork is a very large Directed Acyclic Graph (DAG) whose subgraphs are different candidate neural networks. Thus, in a supernetwork, the weights are shared among a large number of different sub-architectures that have edges in common, each of which is considered as a path within the supernet. The essential idea is to train one supernetwork that spans many options for the final design rather than generating and training thousands of networks independently. In addition to the learned parameters, a set of architecture parameters are learnt to depict preference for one module over another. Such methods reduce the required computational resources to only a few GPU days. More recent works further combine this weight-sharing paradigm, with a continuous relaxation of the search space, which enables the use of gradient-based optimization methods. These approaches are generally referred to as differentiable NAS and have proven very efficient in exploring the search space of ne
Privacy Lost
Privacy Lost is a 2023 short science fiction film directed by Peter Stoel and Robert Berger. It follows a family using augmented reality (AR) and artificial intelligence (AI) devices capable of reading emotional states, raising questions about privacy and manipulation. == Premise == Privacy Lost follows a family using AR glasses that capture and interpret emotions in real time. As the parents argue in a restaurant, their emotional states and even hidden feelings become visible through these glasses. An AI-driven waiter adapts its appearance for each family member, employing emotional data to influence their decisions. == Cast == Brian Kant as Waiter Michael Krass as Husband Estelle Levinson as Waitress Thor van der Linden as Scotty Carlijn van Ramshorst as Wife == Production == Filming took place at HeadQ Productions, a virtual studio located in Amsterdam. The creators sought to depict a near-future scenario in which real-time emotion analysis becomes part of daily interactions. The film was screened at the Augmented World Expo (AWE), where it was noted for its thematic focus on AI-driven manipulation and emotional tracking. The depiction of AR glasses and AI characters integrates modern visual effects to show how devices might analyze emotional responses in real time. It also depicts how AI-driven interactions could influence consumer decisions, pointing to concerns over potential misuse. == Themes == Privacy Lost focuses on the intersection of advanced AI capabilities and AR environments, showing how real-time emotional analysis can be leveraged for targeted persuasion. The film aims to highlight the social and ethical implications of emerging AR and AI technologies, underlining how establishing clear regulatory frameworks for them is necessary to protect individual privacy, govern the storage of emotion-based data, and prevent manipulative practices. Critics describe the film’s theme as dystopian and note that such a reality is unlikely to occur in the near future. However, despite the exaggerated scenario, the film emphasizes the importance of a responsible approach by developers toward emerging technologies.
Deepfake
Deepfakes (a portmanteau of 'deep learning' and 'fake') are images, videos, or audio that have been edited or generated using artificial intelligence, AI-based tools or audio-video editing software. They may depict real or fictional people and are considered a form of synthetic media, that is media that is usually created by artificial intelligence systems by combining various media elements into a new media artifact. While the act of creating fake content is not new, deepfakes uniquely leverage machine learning and artificial intelligence techniques, including facial recognition algorithms and artificial neural networks such as variational autoencoders and generative adversarial networks (GANs). In turn, the field of image forensics has worked to develop techniques to detect manipulated images. Deepfakes have garnered widespread attention for their potential use in creating child sexual abuse material, celebrity pornographic videos, revenge porn, fake news, hoaxes, bullying, and financial fraud. Academics have raised concerns about the potential for deepfakes to promote disinformation and hate speech, as well as interfere with elections. In response, the information technology industry and governments have proposed recommendations and methods to detect and mitigate their use. Academic research has also delved deeper into the factors driving deepfake engagement online as well as potential countermeasures to malicious application of deepfakes. From traditional entertainment to gaming, deepfake technology has evolved to be increasingly convincing and available to the public, allowing for the disruption of the entertainment and media industries. == History == Photo manipulation was developed in the 19th century and soon applied to motion pictures. Technology steadily improved during the 20th century, and more quickly with the advent of digital video. Deepfake technology has been developed by researchers at academic institutions beginning in the 1990s, and later by amateurs in online communities. More recently, the methods have been adopted by industry. The development of generative adversarial networks (GANs) in the mid-2010s represented a key technical turning point in the evolution of deepfakes. GANs allowed for the creation of highly realistic fake images and videos by training competing neural networks, achieving a much improved visual fidelity over previous methods of creating the content using rules or by using autoencoders, and formed the basis for modern deepfake methods. === Academic research === Academic research related to deepfakes is split between the field of computer vision, a sub-field of computer science, which develops techniques for creating and identifying deepfakes, and humanities and social science approaches that study the social, ethical, aesthetic implications as well as journalistic and informational implications of deepfakes. As deepfakes have risen in prominence in popularity with innovations provided by AI tools, significant research has gone into detection methods and defining the factors driving engagement with deepfakes on the internet. Deepfakes have been shown to appear on social media platforms and other parts of the internet for purposes ranging from entertainment and education related to deepfakes to misinformation to elicit strong reactions. There are gaps in research related to the propagation of deepfakes on social media. Negativity and emotional response are the primary driving factors for users sharing deepfakes. === Social science and humanities approaches to deepfakes === In cinema studies, deepfakes illustrate how "the human face is emerging as a central object of ambivalence in the digital age". Video artists have used deepfakes to "playfully rewrite film history by retrofitting canonical cinema with new star performers". Film scholar Christopher Holliday analyses how altering the gender and race of performers in familiar movie scenes destabilizes gender classifications and categories. The concept of "queering" deepfakes is also discussed in Oliver M. Gingrich's discussion of media artworks that use deepfakes to reframe gender, including British artist Jake Elwes' Zizi: Queering the Dataset, an artwork that uses deepfakes of drag queens to intentionally play with gender. The aesthetic potentials of deepfakes are also beginning to be explored. Theatre historian John Fletcher notes that early demonstrations of deepfakes are presented as performances, and situates these in the context of theater, discussing "some of the more troubling paradigm shifts" that deepfakes represent as a performance genre. While most English-language academic studies of deepfakes focus on the Western anxieties about disinformation and pornography, digital anthropologist Gabriele de Seta has analyzed the Chinese reception of deepfakes, which are known as huanlian, which translates to "changing faces". The Chinese term does not contain the "fake" of the English deepfake, and de Seta argues that this cultural context may explain why the Chinese response has centered on practical regulatory measures to "fraud risks, image rights, economic profit, and ethical imbalances". === Computer science research on deepfakes === A landmark early project was the "Video Rewrite" program, published in 1997. The program modified existing video footage of a person speaking to depict that person mouthing the words from a different audio track. It was the first system to fully automate this kind of facial reanimation, and it did so using machine learning techniques to make connections between the sounds produced by a video's subject and the shape of the subject's face. Contemporary academic projects have focused on creating more realistic videos and improving deepfake techniques. The "Synthesizing Obama" program, published in 2017, modifies video footage of former president Barack Obama to depict him mouthing the words contained in a separate audio track. The project lists as a main research contribution to its photorealistic technique for synthesizing mouth shapes from audio. The "Face2Face" program, published in 2016, modifies video footage of a person's face to depict them mimicking another person's facial expressions. The project highlights its primary research contribution as the development of the first method for re-enacting facial expressions in real time using a camera that does not capture depth, enabling the technique to work with common consumer cameras. Researchers have also shown that deepfakes are expanding into other domains such as medical imagery. In this work, it was shown how an attacker can automatically inject or remove lung cancer in a patient's 3D CT scan. The result was so convincing that it fooled three radiologists and a state-of-the-art lung cancer detection AI. To demonstrate the threat, the authors successfully performed the attack on a hospital in a White hat penetration test. A survey of deepfakes, published in May 2020, provides a timeline of how the creation and detection of deepfakes have advanced over the last few years. The survey identifies that researchers have been focusing on resolving the following challenges of deepfake creation: Generalization. High-quality deepfakes are often achieved by training on hours of footage of the target. This challenge is to minimize the amount of training data and the time to train the model required to produce quality images and to enable the execution of trained models on new identities (unseen during training). Paired Training. Training a supervised model can produce high-quality results, but requires data pairing. This is the process of finding examples of inputs and their desired outputs for the model to learn from. Data pairing is laborious and impractical when training on multiple identities and facial behaviors. Some solutions include self-supervised training (using frames from the same video), the use of unpaired networks such as Cycle-GAN, or the manipulation of network embeddings. Identity leakage. This is where the identity of the driver (i.e., the actor controlling the face in a reenactment) is partially transferred to the generated face. Some solutions proposed include attention mechanisms, few-shot learning, disentanglement, boundary conversions, and skip connections. Occlusions. When part of the face is obstructed with a hand, hair, glasses, or any other item then artifacts can occur. A common occlusion is a closed mouth which hides the inside of the mouth and the teeth. Some solutions include image segmentation during training and in-painting. Temporal coherence. In videos containing deepfakes, artifacts such as flickering and jitter can occur because the network has no context of the preceding frames. Some researchers provide this context or use novel temporal coherence losses to help improve realism. As the technology improves, the interference is diminishing. Overall, deepfakes are expected to have several implications in media and society, med
Colors!
Colors! is a series of digital painting applications for handheld game consoles and mobile devices. Originally created as a homebrew application for Nintendo DS (as Colors!), which was since legitimately distributed on PlayStation Vita, iOS, and Android, the project eventually evolved into an officially licensed application for Nintendo 3DS (as Colors! 3D) and Nintendo Switch (as Colors Live). == History == === Colors! === Colors! was originally released in June 2007 as a simple homebrew painting application for the Nintendo DS. It was developed by Jens Andersson, a programmer and designer on sabbatical from the games industry who wanted to experiment with the potential of the new handheld platform. Shortly after, Rafał Piasek created an online gallery where users could upload paintings made with the program. Colors! quickly became one of the best-known homebrew applications on the Nintendo DS, and in September 2008, it was also released for the iPhone and iPod Touch. As of August 2010, it had been downloaded almost half a million times. It was voted the most popular homebrew application on the Nintendo DS by readers of the R4 for DS blog. Development of Colors! DS homebrew officially ended in December 2010 although the official gallery still accepted submissions from DS users until 2020 when Colors! Gallery was discontinued. === Colors! 3D === Colors! 3D is a successor to the application Colors! for the Nintendo 3DS. It was released as an officially licensed application for the Nintendo eShop in North America on April 5, 2012, and in the PAL region on April 19, 2012. It was later released in Japan on August 21, 2013, published by Arc System Works. Colors! 3D allows users to draw on five layers, each on their own stereoscopic 3D plane. Drawing is done on the bottom screen, while the top screen displays the painting in 3D. While drawing, players can use the various controls on the Nintendo 3DS to change layers, zoom and pan, and alter the pressure of their brush. Pressing the L button allows users to access a menu to change brush type, size, and opacity, modify the layers, use the camera to provide references, and more. When the user finishes their painting, they can export it to the SD card for viewing in the Nintendo 3DS Camera application. Users can also upload their finished creations to an online gallery, viewed on the 3DS or the official website. Gallery features include hashtags and the ability to follow artists and post comments. Each painting also features a replay feature that allows viewers to see how it was drawn. The application also features local multiplayer, allowing several people to work cooperatively on a painting. In April 2024, the developers of Colors! 3D collaborated with the Pretendo Network project to officially add support for the application, meaning Colors! 3D will continue to operate as normal when using Pretendo Network. ==== Reception ==== IGN gave the application a score of 9.0 and an Editor's Choice award, praising its simple interface and tutorials. Destructoid gave the app a 9.0, calling it "a simple and incredibly fun tool with an amazing community of artists proudly displaying their beautiful and funny 3D images." Nintendo Life gave the app a 9/10, stating, "Though lacking in any structured play, Colors! 3D’s robust free drawing system and unique ability to let anyone create their own three-dimensional artwork more than make up for this." === Colors Live === A Nintendo Switch successor called Colors Live (stylised as Colors L!ve) was released in 2020 after being funded via a Kickstarter campaign. This expanded upon the features of previous installments by adding new brushes, increasing the maximum number of layers to ten, and introducing blend modes. A new game mode called Colors Quest was also included. A pressure-sensitive pen called the Colors SonarPen was developed in collaboration with GreenBulb to facilitate drawing on the Nintendo Switch, and comes pre-bundled with physical copies of the game. ==== Colors Quest ==== This new mode acts as a story-driven adventure wherein players are given a daily drawing challenge with a specific theme and certain stipulations that must be fulfilled. Once the drawing is complete, players must anonymously score other players' submissions, these scores are then aggregated to produce a personal ranking that measures the improvement in the player's art skills over time.
Blackboard system
A blackboard system is an artificial intelligence approach based on the blackboard architectural model, where a common knowledge base, the "blackboard", is iteratively updated by a diverse group of specialist knowledge sources, starting with a problem specification and ending with a solution. Each knowledge source updates the blackboard with a partial solution when its internal constraints match the blackboard state. In this way, the specialists work together to solve the problem. The blackboard model was originally designed as a way to handle complex, ill-defined problems, where the solution is the sum of its parts. == Metaphor == The following scenario provides a simple metaphor that gives some insight into how a blackboard functions: A group of specialists are seated in a room with a large blackboard. They work as a team to brainstorm a solution to a problem, using the blackboard as the workplace for cooperatively developing the solution. The session begins when the problem specifications are written onto the blackboard. The specialists all watch the blackboard, looking for an opportunity to apply their expertise to the developing solution. When someone writes something on the blackboard that allows another specialist to apply their expertise, the second specialist records their contribution on the blackboard, hopefully enabling other specialists to then apply their expertise. This process of adding contributions to the blackboard continues until the problem has been solved. == Components == A blackboard-system application consists of three major components The software specialist modules, which are called knowledge sources (KSs). Like the human experts at a blackboard, each knowledge source provides specific expertise needed by the application. The blackboard, a shared repository of problems, partial solutions, suggestions, and contributed information. The blackboard can be thought of as a dynamic "library" of contributions to the current problem that have been recently "published" by other knowledge sources. The control shell, which controls the flow of problem-solving activity in the system. Just as the eager human specialists need a moderator to prevent them from trampling each other in a mad dash to grab the chalk, KSs need a mechanism to organize their use in the most effective and coherent fashion. In a blackboard system, this is provided by the control shell. === Learnable Task Modeling Language === A blackboard system is the central space in a multi-agent system. It's used for describing the world as a communication platform for agents. To realize a blackboard in a computer program, a machine readable notation is needed in which facts can be stored. One attempt in doing so is a SQL database, another option is the Learnable Task Modeling Language (LTML). The syntax of the LTML planning language is similar to PDDL, but adds extra features like control structures and OWL-S models. LTML was developed in 2007 as part of a much larger project called POIROT (Plan Order Induction by Reasoning from One Trial), which is a Learning from demonstrations framework for process mining. In POIROT, Plan traces and hypotheses are stored in the LTML syntax for creating semantic web services. Here is a small example: A human user is executing a workflow in a computer game. The user presses some buttons and interacts with the game engine. While the user interacts with the game, a plan trace is created. That means the user's actions are stored in a logfile. The logfile gets transformed into a machine readable notation which is enriched by semantic attributes. The result is a textfile in the LTML syntax which is put on the blackboard. Agents (software programs in the blackboard system) are able to parse the LTML syntax. == Implementations == We start by discussing two well known early blackboard systems, BB1 and GBB, below and then discuss more recent implementations and applications. The BB1 blackboard architecture was originally inspired by studies of how humans plan to perform multiple tasks in a trip, used task-planning as a simplified example of tactical planning for the Office of Naval Research. Hayes-Roth & Hayes-Roth found that human planning was more closely modeled as an opportunistic process, in contrast to the primarily top-down planners used at the time: While not incompatible with successive-refinement models, our view of planning is somewhat different. We share the assumption that planning processes operate in a two-dimensional planning space defined on time and abstraction dimensions. However, we assume that people's planning activity is largely opportunistic. That is, at each point in the process, the planner's current decisions and observations suggest various opportunities for plan development. The planner's subsequent decisions follow up on selected opportunities. Sometimes, these decision-sequences follow an orderly path and produce a neat top-down expansion as described above. However, some decisions and observations might also suggest less orderly opportunities for plan development. A key innovation of BB1 was that it applied this opportunistic planning model to its own control, using the same blackboard model of incremental, opportunistic, problem-solving that was applied to solve domain problems. Meta-level reasoning with control knowledge sources could then monitor whether planning and problem-solving were proceeding as expected or stalled. If stalled, BB1 could switch from one strategy to another as conditions – such as the goals being considered or the time remaining – changed. BB1 was applied in multiple domains: construction site planning, inferring 3-D protein structures from X-ray crystallography, intelligent tutoring systems, and real-time patient monitoring. BB1 also allowed domain-general language frameworks to be designed for wide classes of problems. For example, the ACCORD language framework defined a particular approach to solving configuration problems. The problem-solving approach was to incrementally assemble a solution by adding objects and constraints, one at a time. Actions in the ACCORD language framework appear as short English-like commands or sentences for specifying preferred actions, events to trigger KSes, preconditions to run a KS action, and obviation conditions to discard a KS action that is no longer relevant. GBB focused on efficiency, in contrast to BB1, which focused more on sophisticated reasoning and opportunistic planning. GBB improves efficiency by allowing blackboards to be multi-dimensional, where dimensions can be either ordered or not, and then by increasing the efficiency of pattern matching. GBB1, one of GBB's control shells implements BB1's style of control while adding efficiency improvements. Other well-known of early academic blackboard systems are the Hearsay II speech recognition system and Douglas Hofstadter's Copycat and Numbo projects. Some more recent examples of deployed real-world applications include: The PLAN component of the Mission Control System for RADARSAT-1, an Earth observation satellite developed by Canada to monitor environmental changes and Earth's natural resources. The GTXImage CAD software by GTX Corporation was developed in the early 1990s using a set of rulebases and neural networks as specialists operating on a blackboard system. Adobe Acrobat Capture (now discontinued), as it used a blackboard system to decompose and recognize image pages to understand the objects, text, and fonts on the page. This function is currently built into the retail version of Adobe Acrobat as "OCR Text Recognition". Details of a similar OCR blackboard for Farsi text are in the public domain. Blackboard systems are used routinely in many military C4ISTAR systems for detecting and tracking objects. Another example of current use is in Game AI, where they are considered a standard AI tool to help with adding AI to video games. == Recent developments == Blackboard-like systems have been constructed within modern Bayesian machine learning settings, using agents to add and remove Bayesian network nodes. In these 'Bayesian Blackboard' systems, the heuristics can acquire more rigorous probabilistic meanings as proposal and acceptances in Metropolis Hastings sampling though the space of possible structures. Conversely, using these mappings, existing Metropolis-Hastings samplers over structural spaces may now thus be viewed as forms of blackboard systems even when not named as such by the authors. Such samplers are commonly found in musical transcription algorithms for example. Blackboard systems have also been used to build large-scale intelligent systems for the annotation of media content, automating parts of traditional social science research. In this domain, the problem of integrating various AI algorithms into a single intelligent system arises spontaneously, with blackboards providing a way for a collection of distributed, modular natural language processing algorithm