Generative AI exploded for a year and a half： artifact or chicken ribs
Figure /cfp
Wen | Liu Wei (Director, Laboratory of Human-Computer Interaction and Cognitive Engineering, Beijing University of Posts and Telecommunications)
* This article is the author’s participation in the live broadcast of "Nine Schools Roundtable" of Nine Schools News, "Generative AI: Is the Bubble Coming? The speeches on the topic discussion were compiled and reviewed by the author before publication.
The decision-making process of AI is often unexplained, and the result is like a "black box"
ChatGPT improves efficiency in content generation, data analysis and customer service, but it also causes social problems, such as the impact on simple work. Intelligent products, such as radish running, have affected the employment of online taxi drivers and taxi drivers. In the field of education and training, the application of ChatGPT promotes the development of personalized education and knowledge retrieval. However, it has limitations in understanding and reasoning ability, and sometimes it will cause problems such as machine illusion and algorithm jailbreak.
Different from human beings, ChatGPT relies on a large number of data and samples to solve problems, which may lead to data bias and errors, which shows that the technology itself has two sides. Although it has advantages in some aspects, its application in precision industry and national security still needs to be developed. At present, generative artificial intelligence technology performs well in text processing, image recognition and simple reasoning tasks, but data dependence is both its advantages and disadvantages.
Movies and other fields are cautious about the adoption of generative artificial intelligence, and the technology needs to be improved in realizing industrial visual effects. Generally speaking, generative artificial intelligence benefits from big data, but it is also limited by it. We predict that future intelligent systems may combine data of different scales to achieve wider applications.
Data can not only be generated, but also its usage is constantly changing. Humans depend not only on data, but also on information, knowledge and many non-data elements. At present, generative artificial intelligence faces challenges in data processing, especially at the input end. It relies too much on data and ignores human subjectivity and value outside data. This tendency of "data-only theory" has aroused our concern, because it can only solve some problems.
The popularity of generative AI has not yet reached expectations because:
First of all, the real-time performance is insufficient. When dealing with real-time problems and dynamic scenes, there are problems of delay or insufficient accuracy.
Secondly, the issue of trust. Many people are skeptical about the output of AI, especially when making sensitive or important decisions. The decision-making process of AI is often unexplained, and the result is like a "black box".
Third, the integration problem. When trying to integrate generative AI with other technologies, such as machinery, automation systems and information products, it is often difficult to achieve organic integration, and additional interfaces and technical adjustments are needed, which may lead to many contradictions.
Finally, privacy and data security issues. Problems such as data leakage and personal information protection limit people’s acceptance of generative AI.
When we discuss artificial intelligence, data security has always been the focus of discussion.
Americans believe that only China and the United States can have an in-depth dialogue on artificial intelligence. One of the core problems of artificial intelligence is data, including data collection, cleaning, pollution and falsification, which may affect the authenticity of audio, video and images. If the data problem is misleading in an important period, such as the election, it may lead to unpredictable consequences.
In his book Artificial Intelligence published before his death, Dr. Kissinger particularly emphasized the importance of data problems. He believes that artificial intelligence technology is different from nuclear weapons, and it can be mastered by small countries or small companies, thus affecting big countries and even the whole world. Data is the first level of artificial intelligence and the most dangerous place.
The data of the large model is marked to the feedback of the use process, and the wisdom of human beings runs through it.
The development of large-scale model mainly involves two aspects: first, the interaction with human beings, including correction, labeling and use; Secondly, the technical level, especially the attention model based on Transformer framework. This model is a multi-layer neural network system, and its data quality problems and deviations often come from the linear function and activation function in the neural network, as well as their random deviations. This means that all large models have certain limitations in technical essence.
The so-called "black box" problem, that is, the model is not interpretable, is a characteristic of multi-inner neural network system. Professor Marcus of new york University foresaw this in his book Algebraic Brain more than 20 years ago, which is also the origin of what we now call machine illusion. Therefore, data quality, including the deviation of simulation data, is an urgent problem to be solved.
There are two ways to solve these problems: one is to use gray box or white box technology to improve the transparency and controllability of the model; The second is to introduce human feedback in the process of using the model and optimize the model by strengthening the learning mechanism. Generative artificial intelligence is not only an AI system, but also a complex system involving human-computer interaction. From the initial labeling of data to the feedback in the use process, human wisdom always runs through it.
The effect of generative AI is closely related to its application environment. In certain fields such as finance, specially trained AI can guarantee the quality and diversity of data. With continuous optimization and iteration, the application effect of AI in smart cities, smart transportation, smart education and other fields will gradually improve.
At present, the multi-modal model is developing rapidly, which combines text, image, video, audio and other data forms to provide more accurate application results. These models enhance the understanding and generation ability of content by integrating information from different modes. For example, GPT-4 combines the ability of image and text processing, and models like Sora focus on video generation. Recently, however, due to the complexity of technology, the high demand for computing resources, and the uncertainty of market changes, the early investment and development became unsustainable, which made it difficult for some models to operate normally. Part of the reason why Sora’s information becomes scarce is that there are imperfections in its multimodal characteristics.
Multi-modal model should be combined with single modal model, rather than simply emphasizing multi-modal or single modal. Combining the advantages of the two may be the future development trend.
American experts say that the advantages of the United States are software and hardware, which can "choke others’ necks."
When discussing the advantages and disadvantages of China and the United States in the field of artificial intelligence, artificial intelligence experts from some top institutions in the United States clearly pointed out their respective characteristics.
American experts believe that the advantages of the United States lie in software and hardware, and they even vividly say that these advantages can "choke others’ necks." The advantage of China lies in the richness of data, especially high-quality data. Take the manufacturing industry as an example. As a big manufacturing country in the world, China has the most perfect and complete manufacturing industry chain. With the support of big data, China is in a leading position in model training and optimization. China’s extensive market demand has promoted the richness and innovation of application scenarios, which is something that the United States feels can’t keep up with.
However, realistically speaking, in the field of generative artificial intelligence, we do lag behind the United States in three aspects: first, the corpus, in which many large model companies in China use English content, occupies a huge proportion; Secondly, the software framework, such as Transformer, is basically American technology; The third is hardware, especially GPU. There are relatively few enterprises with a large number of GPUs in China. Although Huawei is also developing, it still lags behind the United States in performance and efficiency.
We realize that there is still a big gap with the United States in software, hardware and corpus. Simple single competition may have fallen behind, and the gap seems to be widening. However, if we can make comprehensive use of data, applications and the advantages of various industries, we may better form an ecosystem. Using ecological advantages, we can make up for the shortcomings in the above three aspects to some extent.
Of course, we are also trying to catch up and need to learn from each other’s advantages. How to organically combine the four elements of software and hardware, data and application is an important problem that we need to solve in the future.
At present, the United States is actively developing large-scale models and infrastructure, such as the constantly upgraded high-performance computing cards such as H100 and H200 Blackware, and the continuous optimization iteration of software. The US government is also formulating policies to impose technology blockade on countries including China, restrict the export of high-performance computing cards, and gradually impose restrictions on software, whether open source or closed source, in an attempt to hinder China’s development in the field of generative artificial intelligence through these measures.
However, China has an important driving force in the development of generative artificial intelligence-demand. Especially the needs of industry. As we have witnessed in the Internet, smart phones, automobiles, shipbuilding and many other fields, the development speed of China is very rapid, and the field of generative artificial intelligence is expected to achieve rapid follow-up.
At present, I think China needs to do more in-depth and practical work in scene segmentation and innovation. Relying solely on AI technology is not comprehensive enough, because the whole industry needs not only AI, but also the comprehensive development of automation, mechanization, informationization and digitalization. This means that we need to consider building an integrated ecosystem.
Orientals often adopt a holistic and systematic perspective when solving problems, while the west prefers the reductionism of science and technology. How to combine these two advantages and make a breakthrough in the field of generative artificial intelligence is a problem worthy of our deep thinking and continuous exploration. By integrating the advantages of the East and the West, we can achieve more comprehensive and in-depth development in this field.
Of course, there may be differences between China and the United States in the development path of generative AI. The United States pays more attention to large-scale and complex investment, while China may pay more attention to the application of models in the industry, and reduce costs by combining automation technology. Such differences may provide different inspirations and ideas for the development of the two countries in the field of AI.
When will the high-cost investment-generated AI be able to return to its original cost?
At present, the profit model of generative artificial intelligence is firstly subscription service, and users access advanced functions or enhanced models by paying subscription fees; Secondly, API charges, providing interfaces for developers to use, and charging according to the number of calls, such as OpenAI’s API; The third is to provide customized AI solutions or consulting services to tailor services for enterprises. These modes together constitute the business ecology of generative artificial intelligence.
Advertising recommendation is another application field of generative artificial intelligence. Although the content generation of personalized advertisements and recommendations has not been widely popularized at present, some enterprises are actively exploring this field. However, this technology still faces some challenges, including sustainability, high cost and technical complexity, which may affect its long-term profitability.
User privacy is also a big problem. Neither OpenAI nor other large model companies have found a perfect solution.
In addition, content quality control is also a thorny challenge. The accuracy and quality assurance of generated content is an urgent problem to be solved at present.
Nevertheless, large-scale models still have obvious advantages in the field of generative artificial intelligence. They can provide flexible services, meet the needs of different users and make personalized customization. In addition, these models are scalable and can serve users on a large scale, thus reducing the unit cost. They also have the ability of continuous innovation and can open up new markets and new businesses.
Regarding the cost of generative artificial intelligence, the first is the operation and maintenance cost, including manpower and R&D expenditure, which is the main source of expenses. Microsoft’s huge investment in OpenAI is evidence. Behind the tens of billions of dollars, there are a series of high expenses such as the salary and recruitment of R&D personnel. Data collection and processing, as well as computing resources and infrastructure construction, also account for a large part of the cost. Imagine that huge computing power demand, whether it is GPU or TPU cluster, has put great pressure on the cost. The process of data collection and processing not only involves complex authorization, cleaning and purchase, but also faces the challenge of massive data storage. The research and application of efficient storage solutions is quite expensive.
The high-performance hardware and data storage solutions in the data center need huge investment to support them.
The cost of operation and maintenance can not be ignored. The continuous work of troubleshooting and system monitoring is the cornerstone to ensure the smooth progress of the project. The surge in electricity costs has made giants such as OpenAI begin to consider building nuclear power plants to meet demand, which shows its importance.
There is also the security issue. The departure of Edia and others is a profound question about the security of the big model. The negligence of this link will undoubtedly further aggravate the burden of cost.
In addition, changes in the external environment, such as market fluctuations and technological progress, will also have a significant impact on costs and profitability.
Although the progress of management and technology helps to reduce some costs, with the increasing scale and complexity of AI, the overall cost remains high. Before making a profit, it is necessary to continuously invest huge amounts of money. This is particularly evident in the United States, while in China, due to the development of landing applications and industry-specific models, the cost may be relatively low.
The return on investment of generative AI is a complex problem in any technical field, especially in the cutting-edge technology such as generative artificial intelligence, which involves many factors:
First of all, technology maturity is a key factor, which determines the cycle length of investment return. New technologies, such as ChatGPT and generative artificial intelligence, usually take a long time to develop, verify, optimize and popularize. These processes are not only time-consuming, but also full of uncertainty, which affects the speed of technology maturity.
Secondly, market acceptance, especially users’ acceptance of new technologies. Taking Rapunzel as an example, it not only touches a wide range of social issues, such as the potential impact of employment structure, but also indirectly affects the promotion and popularization process of generative artificial intelligence and other AI technologies. Furthermore, when it comes to infrastructure construction, such as energy center, computing center and even chip research and development, it often takes time for the achievements in these fields to appear, which tests our patience and long-term vision.
Regarding the cost-effectiveness of generative AI, it is closely linked with the pace of technological progress. With the continuous iteration and optimization of technology, the cost is expected to be gradually reduced, while the benefit is correspondingly improved, which involves the dual consideration of cost structure optimization and benefit growth.
NVIDIA’s stock performance is undoubtedly one of the focuses in the field of generative artificial intelligence, and its fluctuation is directly related to core issues such as cost-effectiveness, investment return cycle, etc.
Furthermore, the implementation of technology has a significant impact on the performance of the stock market in the short term. It is not only the touchstone of technology maturity, but also the vane of market confidence.
The development cycle of this kind of technology is long, which needs the continuous attention and investment from all walks of life. Looking back at the growth process of OpenAI, the support of Microsoft and Musk for many years, and countless attempts and accumulations have made today’s achievements. In contrast, although China’s venture capital environment is gradually improving, it still faces many challenges, such as differences in concepts and poor mechanisms, which limits the emergence of companies like OpenAI to some extent.
Therefore, when we talk about the time period of generative AI, we are actually talking about a risk period. This cycle is full of uncertainty and unpredictable. Although the progress of management and technology can reduce the cost to a certain extent, the scale and complexity of AI are increasing, and the overall cost remains high.
There is no convincing case of commercial application of generative artificial intelligence in the market.
Generative artificial intelligence usually relies on statistical probability to predict the probability of the next word or token in reasoning, but this is only a part of intelligence. Real intelligence also includes many dimensions such as management, psychology, ethics and jurisprudence. Technical maturity may be a major bottleneck for generative artificial intelligence.
In commercial applications, generative artificial intelligence also faces some challenges, including the feasibility of the solution, cost-effectiveness, market demand and application landing scenarios. These complex problems are interrelated and increase the complexity of the whole field. There are no convincing success stories in the market.
If the cost of generative artificial intelligence is to be reduced, the prerequisite is that the emergence of new software architecture and algorithms will help reduce the cost. The improvement of data quality will greatly reduce the cost of data collection and cleaning. With the continuous progress of hardware technology, the price will gradually decrease. For example, the future H100 and H200 may become as cheap as the current CPU. The high labor cost is expected to drop sharply in the future with a better grasp of technology. The cost of the product or system will also be reduced. In the future, breakthroughs in new energy and power technologies, such as nuclear energy and nuclear fusion, may also lead to further cost reduction.
In the short term, factors such as technology maturity, application penetration rate and investment cost have great influence on generative artificial intelligence, which may cause market hesitation and wandering. But in the long run, the potential of artificial intelligence is still huge, and it is expected to bring remarkable economic growth through business model innovation, scale effect and technological synergy.
The comprehensive consideration and evaluation of this technology should combine short-term, medium-term and long-term different effects, and pay attention to the use of industry differences. For example, in the pharmaceutical field, AI has brought about significant changes in protein structure prediction. In the entertainment industry, the significant influence of AI has also appeared in the fields including short video. In the future, there may be major breakthroughs in long videos, film industry, smart tourism, smart cities, smart education and medical fields, such as surgery-assisted decision-making systems and personal assistants. However, I think that the combination of technology and people and industry may be the fastest breakthrough in the short term.
There are indeed similarities between the current generative AI and the Internet bubble period. Both of them are in the initial stage of technological development and have great potential and market expectations, but they are not yet fully mature or widely used. The enthusiasm and high expectations of investors may lead to short-term fluctuations and overestimation. However, with the development of technology and the expansion of application, these bubbles tend to be stable and bring about sustainable change and development.
Machines can’t replace human’s "calculating" ability.
Goldman Sachs’ analysis emphasizes the high cost of generative artificial intelligence in replacing low-paid jobs. Traditionally, technological transformation promotes economic growth by reducing production costs and expanding application scope. However, the high cost of generative artificial intelligence limits its potential in replacing low-paid jobs. The report has reservations about generative artificial intelligence and is not entirely optimistic, which is consistent with the basic concepts of most people, including some industry leaders. This cautious attitude is understandable because it has not been widely used.
From the perspective of rationality, the cost-effectiveness of generative artificial intelligence does exist problems. The current economic environment requires that technology investment can bring returns and be highly utilitarian. However, generative artificial intelligence has not yet shown clear and convincing results. The scientific research field may be more suitable for the breeding of this technology, but it is risky to put it directly into the market, because people’s expectations for it may be too high and even misleading.
It is misleading that generative artificial intelligence is only a branch of artificial intelligence and does not represent the whole field. If a lot of resources are invested in this field, other potential research directions may be ignored. This may lead to going in the wrong direction,
If there is a problem with generative artificial intelligence, how should we remedy it?
In human-computer interaction, it is difficult for generative artificial intelligence to replace human cognitive reasoning. Human cognition and machine computing are two different concepts. The decision-making process of human beings involves the combination of sensibility and rationality, while machines mainly rely on calculation. Although machines have advantages in computing speed, accuracy and storage capacity, human "computing" abilities, such as small sample learning, fuzzy logic and cross-domain thinking, are irreplaceable by machines.
The core of generative artificial intelligence is statistical probability, which is still a mathematical tool, whether based on rules or statistical probability. In human cognitive reasoning, there are dialectical, fuzzy and holistic ways of thinking, and even reversible thinking, which are difficult for machines to achieve at present. In the foreseeable future, it is difficult for generative artificial intelligence to replace human beings unless there are subversive discoveries in the fields of mathematics, physics and psychology, which can formalize human calculations and calculations.
In the existing system of mathematics, physics and human knowledge, it is still very difficult to achieve some goals.
First of all, we need to distinguish three concepts: intelligence, intelligence and wisdom. When we talk about human intelligence, we usually refer to learning ability. For example, some students in 985 and 211 universities have strong learning ability, but this does not mean that they are highly intelligent. In all walks of life, we can see many masters and experienced people. They may not have much intelligence, but they are very "intelligent" because they are good at using data, algorithms, computing power and knowledge. This ability to use makes intelligence higher than intelligence to some extent.
Human beings have a more advanced ability, that is wisdom. Wisdom has strong creativity and needs rich imagination and creativity. For example, the story of a blessing in disguise embodies wisdom, which can see the essence through phenomena and grasp the internal connection of things. The next generation of intelligence, it can’t produce general artificial intelligence (AGI) or super artificial intelligence, and the existing tools can’t do this. This is the result of my discussion with many experts and scholars. We all agree that no matter the United States or other countries, in the foreseeable future, human beings will not be able to create AGI, which is beyond the scope of science fiction, science and technology and popular science.
However, in the foreseeable future, there may be a very powerful form of intelligence, which is different from both humans and machines, but the intelligence of man-machine integration and environment combination. This kind of intelligence can combine the advantages of man and machine, adapt to the environment and cooperate with all forces. This kind of adaptive intelligence may appear, because both human and machine have environmental adaptability, and as long as there is a suitable environment, they will continue to train and run in, thus producing this kind of intelligence of man-machine environment integration.
In different times, the AI convergence moment may be similar to the iPhone moment, which will bring about a revolutionary impact. For example, in the field of neuroscience, the impulsive neural network system poses a great challenge to the existing neural network, which is closer to the working mode of the human brain. Humanoid robots, driverless technology and human-computer collaboration all reflect the trend of transferring the best human abilities to machines. Among them, a key concept is "situational awareness", which involves predicting trends from States and refining perception from feelings.
In the technical route of generative AI, the process from state to potential and from feeling to knowing is the core. However, human beings go further and can engage in the state, from knowledge to feeling, which is difficult for AI to achieve. This requires prior knowledge, intuition, imagination and experience. Therefore, it is a very interesting topic in the future how to integrate situational awareness with the sense of state of affairs to form a two-way intelligent system. This involves not only mathematics and logic, but also children’s cognition, not adults’ cognitive style, because children’s cognition is more flexible and innovative.
We think that the next generation of intelligence may be human-machine environment system intelligence, which is also in line with the characteristics of oriental intelligence. Western intelligence mainly focuses on science and technology, and adopts reductionism to build machines. Oriental intelligence pays more attention to overall planning, and an important feature of oriental thinking is complexity.
If science and technology have replaced religion and transformed the world to a certain extent, it will be a complex system that may replace science and technology in the future. Complex systems include not only science and technology, but also non-science and technology, such as humanities, arts and philosophy, which have a great influence on complex systems.
[Source: Jiupai News]