With the popularity of ChatGPT, it took people by surprise that we know an artificial intelligence tool that can answer questions, tell stories, produce essays and even write code. Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is an important driving force for a new round of technological revolution and industrial transformation. So what the brief history of AI development?
The AI Be Pregnant With
1. Mathematical logic - Symbolism School
1) In the initial stage of mathematical logic, Aristotle (384-322BC), created the deductive method and proposed the general principle of deductive reasoning - syllogism.
2) In the period of logical algebra, Leibniz (1646-1716, German mathematician and philosopher) symbolized formal logic and laid the foundation of mathematical logic.
2. Artificial neural network - Connectionism School
1) In 1890, James, an American biologist, first clarified the structure and function of the human brain, as well as the laws of memory, learning, and association-related functions.
2) In 1943, American neurophysiologists McCloch and Pitts built the first neural network model (MP model).
3) In 1949, the Canadian psychologist proposed the Hebb rule to change the connection strength of the neural network.
3. Behaviorist School
Wiener (1874-1956): A famous American mathematician and founder of cybernetics. In 1948, he created cybernetics. And the influence of it on AI has formed the school of behaviorism.
4. The carrier of AI - Computer
American mathematicians John Mauchly and J. Presper Eckert successfully developed the first general-purpose computer ENIAC on February 14, 1946, which can complete 5000 additions and 400 multiplications per second. Its invention laid the material foundation for AI research.
5. Alan Mathison Turing (1912-1954): British mathematician
1) In 1936, he created the Turing machine.
2) In 1950, he addressed the problem of AI that a computer could be said to "think" in his book ""Computing Machinery and Intelligence".
3) He designed a Turing test, trying to judge whether a machine is intelligent by asking it to imitate a human to answer certain questions.
4) His idea of machine intelligence is one of the direct origins of AI, known as the "father of AI".
5) In July 2019, the British government announced that Turing would be on the £50 note. The glory is comparable to that of Newton and Darwin.
The Birth of AI (1956)
AI was born at a historic gathering.
Time: Summer 1956
Venue: University of Dartmouth
Purpose: To make computers more smarter, or intelligent.
Sponsor:
J. McCarthy, a young mathematician and computer expert at Dartmouth, and later a professor at MIT.
M.L.Minsky, mathematician and neuroscientist at Harvard University, later a professor at MIT.
N. Lochester, head of the IBM Corporation Information Center
C.E. Shannon, a mathematics researcher in Bell Labs
Participant:
T.more, A.L.Samuel, from IBM Corporation
O.Selfridge, R.Solomonff, from MIT
A. Newell, from RAND Corporation
H.A. Simon, from Carnegie Institute of Technology
Meeting results: The term Artificial Intelligence (AI) was officially adopted by McCarthy.
Formative period (1956 - 1974): Rapid Development
1) In 1956, Samuel successfully developed the checkers program on the IBM computer, which successfully defeated the then chess master Robert Nelly.
2) In 1957, H. A. Simon and Allen Newell developed a mathematical theorem proving program called Logic Theory Machine.
3) In 1960, McCarthy developed the LISP language, which became the most important programming language in the field of AI for decades to come.
4) In 1965, Robinson proposed the resolution (digestion) principle.
5) In 1968, Shakey, the first intelligent robot developed by Stanford Research Institute (SRI), has human-like senses, such as touch and hearing.
The First Trough (1974-1980)
1) Failure to overpredict, causing significant damage to AI.
2) "In 20 years, machines will be able to do everything a man can do."— Simon, 1965
3) "In 3-8 years we will have a computer with the intelligence of an average human being. Such a machine can read Shakespeare, grease cars, play politics, tell jokes, quarrel …Its intellect will be unrivaled.” — Minsky, 1977.
4) Samuel's chess program lost in 1-4 against the world champion at that time.
5) The resolution method has limited capabilities. It couldn’t prove "the sum of two continuous functions is still a continuous function", no matter how many arguments are made.
6) James, a mathematician at the University of Cambridge in the United Kingdom, published a comprehensive report on AI in accordance with the will of the British government, claiming that AI is mediocre even if it is not a scam.
The Golden Age (1980-1987)
1. Expert system
1) The realization of AI from theoretical research to application of specialized knowledge is an important breakthrough and turning point in the history of AI development.
2) In 1976, Feigenbaum developed the MYCN expert system to assist physicians in diagnosing bacterial infections and providing optimal prescriptions.
3) In 1976, Duda and others at Stanford University developed the geological exploration expert system PROSPECTOR.
2. AI was introduced to the market and showed practical value.
1) In 1981, the Japanese Ministry of Economy, Trade and Industry allocated 850 million U.S. dollars to support the 5th-generation computer project. The goal is to create machines that can converse with humans, translate languages, interpret images, and reason like humans.
2) The UK started the £350 million Alvey project.
3) The U.S. Defense Advanced Research Projects Agency ( DARPA ) invested three times as much in AI in 1988 as it did in 1984.
The Second Trough (1987 - 1993)
The initial success of the expert system maintenance costs are high. They are hard to upgrade, hard to use, and fall prey to all sorts of problems that have been exposed. By the late 1980s, the Strategic Computing Initiative slashed funding for AI.
The magnificent "fifth generation project" of the Japanese ten years ago has not been realized. In fact, some of these goals, such as "starting a conversation with people," were not achieved until 2010. As with other AI projects, expectations were much higher than what was really possible.
The Steady Period (1993 - 2011)
1) The research on machine learning, artificial neural network, intelligent robot and behaviorism tended to be in-depth.
2) Intelligent computing made up for the lack of AI in mathematical theory and calculation, updated and enriched the theoretical framework of artificial intelligence, and brought AI into a new development period.
3) In 2000, Honda released the robot product ASIMO. After more than ten years of upgrading and improvement, it is now one of the most advanced robots in the world.
4) In 2011, the AI program "Watson" developed by IBM participated in a quiz show and defeated two human champions.
The Booming Period: 2012-Present
The explosive growth of data has provided sufficient nourishment for AI. Computing platforms such as ubiquitous perception data and graphics processors, and machine deep learning have joined forces to create momentum. AI is ushering in its vigorous development period. Now humanity has officially entered the era of AI.
The Development Status
Special-purpose AI has made breakthrough progress, such as playing Go, climbing stairs, assembling a certain piece of equipment, etc. In these domain-specific or single-task-oriented aspects, AI can surpass human intelligence.
General AI is in its infancy:
The research and application of general AI has a long way to go. The human brain is a general-purpose intelligent system that can draw inferences about other cases from one instance and integrate them comprehensively. It can handle various problems such as vision, hearing, judgment, reasoning, learning, thinking, planning, and design.
At present, there is still a huge gap between AI and human intelligence, because it still has many shortcomings.
"Intelligence + X" has become an innovative model for AI applications:
It is becoming more and more mature, and AI is rapidly infiltrating and integrating into various industries to reshape the development of the entire society. This is the most important manifestation of the fourth technological revolution driven by AI.
AI Future Trends
AI has three elements: data, computing power and algorithms. Data is the raw material of knowledge, and computing power and algorithms provide "computing intelligence" to learn knowledge and achieve specific goals. The technological improvement of AI for more than 60 years can be attributed to the development of algorithms, computing power and data levels.
1. Data
It is the basic element of mapping the real world to build a virtual world. As the amount of data grows exponentially, the territory of the virtual world is also expanding. Unlike open source AI algorithms, key data is often not open. Data privacy and private domainization are a trend. Data for AI applications, just like traffic is the moat of the Internet. Only core data can have key AI capabilities.
Figure 1: Global Data Volume
2. Computing Power
Computing is key to AI, and the wave of deep learning since the 2010s is largely due to advances in computing power.
1) Quantum computing
Because the development of computing chips according to Moore's Law is becoming more and more ineffective, the slowdown in the progress of computing power will limit the future AI technology. Quantum computing provides a new level to enhance computing power. As the number of qubits of a quantum computer increases exponentially, and its computing power is exponential in the number of qubits, this growth rate will be far greater than the amount of data, which brings a powerful hardware base to AI in the era of data explosion.
2) Edge computing
As a supplement and optimization of cloud computing, edge computing is speeding up from the cloud to the edge and into the smaller and smaller IoT devices. So tiny machine learning (TinyML) is stronger to solve issues such as power consumption, latency, and accuracy.
3) Brain-inspired computing
Various computing systems with brain-inspired computing chips as the core are gradually showing their advantages in dealing with certain intelligent problems. The design of brain-inspired computing chips will draw ideas from the design methodology and development history of existing processors, and realize complete hardware functions based on the theory of computational completeness combined with application requirements. At the same time, the brain-inspired computing basic software will integrate the related computing programming language and framework to realize the gradual evolution of the computing system from "special" to "general".
4) The AI computing center
The AI computing center is based on the latest AI theories and adopts a leading computing architecture. It is a comprehensive platform integrating public computing power services, data open sharing, intelligent ecological construction, and industrial innovation, providing the full-stack capabilities of computing power, data and algorithms. In short, it is a new type of computing infrastructure for the rapid development and application of AI. In the future, with the continuous development of an intelligent society, the computing center will become a key information infrastructure, promote the deep integration of the digital economy and traditional industries, accelerate industrial transformation and upgrading, and speed high-quality economic development.
3. Algorithm
1) Auto machine learning (AutoML)
The central question addressed by AutoML include: Which machine learning algorithm to use on a given dataset? Whether and how to preprocess its features, and how to set all hyperparameters? As machine learning has made great strides in many application areas, this has contributed to the growing demand for related systems. With the help of AutoMl and MLOps technologies, the manual training and deployment process of machine learning will be greatly reduced, and technicians can focus on core solutions.
Figure 2: AutoML
2) Evolving in the direction of distributed privacy protection
At present, many countries and regions around the world have introduced data regulatory regulations, such as HIPAA (US Health Insurance Portability and Accountability Act), GDPR (EU General Data Protection Regulation), "Data Security Act", "Personal Privacy Protection Act", etc., through strict Regulations restrict the exchange of private data among multiple agencies. Distributed privacy-preserving machine learning (federated learning) protects the input data of machine learning model training through encryption and distributed storage.
3) Data and mechanism fusion
The development of AI models is easy. Modeling based on data summarizes the laws from the data and pursues the application effect in practice. And modeling based on the mechanism is deduced from the basic physical laws.
A good, mainstream model usually highly summarizes the laws of the data and fits the mechanism. Because it touches the essence of the problem. Just like scientific theories, they are often concise without too many patches, and this solves the problem of convergence speed and generalization at the same time.
4) Development of neural network model structure
The evolution of neural networks has been along the direction of modularization + hierarchy, constantly combining multiple modules that undertake relatively simple tasks.
The neural network structure detects basic features through lower-level modules, and higher-order features at higher levels. Whether it is a multi-layer feedforward network or a convolutional neural network, this modularity. Because the problems we deal with (images, voice, and text) often have natural modularity, if the learning network matches the inherent modularity of the problem itself, better results can be achieved.
Hierarchy is not just the topological superposition of the network, but more importantly, the upgrade of the learning algorithm. Because simply deepening the hierarchy may lead to problems such as the disappearance of the gradient of the BP network.
5) Integration of multi-school methods
Through the integration of multi-school methods, the strengths and weaknesses of complementary algorithms can be achieved. Such as:
a.Bayesian and neural network integration, like Neil Lawrence's Deep Gaussian process, replace the neural network layer with a simple probability distribution.
b.The fusion of symbolism, ensemble learning and neural network.
c.Integration of symbolism and neural network: Integrate the knowledge base into the neural network, such as GNN and knowledge map representation learning.
d.The integration of neural network and reinforcement learning, such as Google's Alpha Go based on DNN+ reinforcement learning, makes AI’s performance approach to that of humans.
6) Large-scale un/selfsupervised pre-training
If intelligence is a cake, the bulk of the cake is unsupervised learning, the icing on the cake is supervised learning, and the cherry on the cake is reinforcement learning (RL) — Yann Lecun.
Supervised learning requires sufficient labeled data. However, manually labeling a large amount of data is time-consuming and laborious. In some fields, it is almost impossible to obtain sufficient labeled data. Utilizing a large amount of unlabeled data in reality through large-scale unsupervised pre-training methods is a research hotspot. AI large models have ultra-large-scale parameters and huge training data, and the general attributes of AI can be improved through the huge quantization of the model, and lower the threshold for the applications. For example, the emergence of GPT-3 has inspired the continued exploration and research of large-scale self-supervised pre-training methods. In the future, the cross-language self-supervised pre-training model based on multi-modal data such as large-scale images, voice, and video will be further developed, and the cognitive and reasoning capabilities of the model will also be improved.
7) The causal-based learning methods
Most of the current AI models focus on the correlation between data features, but the correlation is not equivalent to the more original causal relationship, which may lead to deviations in prediction results, poor ability to resist attacks, and the model often lacks interpretability. In addition, the model requires independent and identically distributed assumptionss. If the test data and training data come from different distributions, the statistical learning model is often ineffective, and for the causal inference studies: how to learn a causal model that can work under different distributions and contains causal mechanisms, and then use the causal model for intervention or counterfactual inference.
8) Explainable AI development
Explainable AI has the potential to be at the heart of the future of machine learning, and as models become more complex, it becomes increasingly difficult to identify simple, explainable rules. It means that the operation of AI is transparent, which is convenient for humans to supervise and accept AI, so as to ensure the fairness, security and privacy of the algorithm.
Figure 3: Explainable AI
The above is more or less one-sided view of AI trends from a technical point of view. Although technology is the primary productive force and has its own development rules, it cannot be ignored that technology serves the market, and only by combining technology with stable market demand can we achieve substantial growth.
Sign in to leave a comment.