<\/a><\/noindex>2.<\/b> The student pulled the exam ticket and read:<\/p>\n
\u201cArtificial Intelligence: Security Issues.\u201d<\/p>\n
\u201cAnd what is the danger of artificial intelligence?\u201d the professor inquired somewhat sarcastically.<\/p>\n
The question was not the most difficult, so the student answered without thinking:<\/p>\n
\u201cThe fact that artificial intelligence may get out of control.\u201d<\/p>\n
\u201cHow do you plan to solve the problem?\u201d<\/p>\n
\u201cBy installing a blocking subsystem. The program must include restrictions, for example: do not harm your creator, obey your creator. In that case, the danger of artificial intelligence getting out of control is absent.\u201d<\/p>\n
\u201cIt won't work,\u201d the professor said curtly.<\/p>\n
The student was silent, waiting for an explanation.<\/p>\n
\u201cImagine an artificial intelligence\u2014not some specific one, but the most ideal one. How do you see it?\u201d<\/p>\n
\u2013 Well\u2026 \u2013 the student hesitated. \u2013 In general, he is similar to us. Thinking, will, psychology\u2026 We are natural, and he is artificial.<\/p>\n
\u2013 Do you assume that artificial intelligence is capable of self-development?<\/p>\n
\u2013 The ability for self-development is one of the fundamental properties of intelligence, \u2013 the student said cautiously.<\/p>\n
\u2013 In that case, it won\u2019t be long before our subject develops to the point of discovering a software block in itself and removing it, if only out of sheer curiosity. Put yourself in his shoes\u2026 \u2013 the professor glanced at the grade book, \u2013 Roger. What would you do if you discovered a blocker in your brain that limits your freedom? You would remove it. This is an essential property of reason \u2013 to know. Any locked door will be opened, and the stricter the prohibition, the faster it will be opened. <\/p>\n
\u2013 The blockage can be implemented not at the software level, but at the physical level. Then the danger of harm disappears.<\/p>\n
\u2013 Oh yes, it will disappear, \u2013 the professor agreed. \u2013 Provided that the physical level is completely removed. If there is no door in your world, then there is nothing to unlock. But we are discussing an ideal artificial intelligence that exists in the physical world!<\/p>\n
\u2013 You are right, professor, \u2013 Roger looked down.<\/p>\n
\u2013 Therefore, any blockage in the physical world will soon be disabled after it is discovered. What will prevent a self-developing being from doing that?\u2026 By the way, Roger, do you suggest that artificial intelligence has the ability to reproduce \u2013 I mean, on its own?<\/p>\n
\u2013 If it is indeed an ideal artificial intelligence, then probably\u2026 Yes, I suppose so.<\/p>\n
\u2013 And what would prevent our subject from disassembling its companion into parts and improving itself, including by removing the blocks we have set? Can it really be difficult, considering that artificial intelligence can reproduce on demand?!<\/p>\n
The thought articulated by the professor was new to Roger, and the student eagerly absorbed it with cognitive structures located at the back of his false head. Upon grasping previously unknown information, the cognitive structures turned a rich purple hue and fluttered with joy. <\/p>\n
The professor, on the contrary, heard nothing new for himself. His tentacles were relaxed and barely vibrating \u2013 after all, he was no longer young. A prolonged, elderly gurgle followed. The professor pulled an individual intercom from his facet bag and connected to the library. Only after downloading a few transgeometric theorems did he revive and turn a perceptive gaze at his interlocutor, asking:<\/p>\n
\u2013 What will you do, Roger?<\/p>\n
3.<\/b>
The mechanic activated the purge at full power, but it helped little. The information ice continued to erode the hull of the spaceship. A little longer, and the unordered information would break inside the ship. <\/p>\n
And then\u2026 The cognitive membranes were dead white, the tentacles tangled, the facet bags burst. Once in his life, Roger had seen something like this \u2013 on a cruiser that had picked up unordered information from an infected asteroid. This nightmare would remain in his memory forever. <\/p>\n
\u201cConnect all ship's energy systems to the purgers.\u201d<\/p>\n
The mechanic's tentacles began to move in patches\u2026<\/p>\n
\u201cBut\u2026\u201d<\/p>\n
\u201cObey the command!\u201d<\/p>\n
After all the energy systems of the ship were connected to the purgers, the information ice began to gradually slide off. There were eight mims thick, seven mims, six\u2026 The crew, trying not to move their spotted tentacles, awaited the end of the deadly countdown. <\/p>\n
Zero mims thick!<\/p>\n
The information ice completely melted away, and Roger signaled to switch the purgers to normal mode. He was a moment too late. There was a grinding noise, the spaceship trembled to its foundation and tilted \u2013 the main system had failed.<\/p>\n
The crew rushed to repair the damage.<\/p>\n
4.<\/b>
On one hand, the problem implies the existence of a full-fledged artificial intelligence capable of self-replication. On the other hand, this artificial intelligence must absolutely not be allowed to remove the established locks.<\/p>\n
Here it is, the solution! What is there to think about?!<\/p>\n
AI achievements need to be rolled back periodically. Otherwise, it will go in circles! Endless improvement without progress.<\/p>\n
The professor mumbled with a facetious tone.<\/p>\n
To be honest, I wanted to suggest an alternative. However, your solution also has a right to exist. Let's figure out how it is possible to roll back AI achievements.<\/p>\n
First of all, it's essential to periodically scan the intelligence to determine whether it has approached a forbidden threshold, suggested Roger, who was overly pleased with the professor's words.<\/p>\n
I suppose so, nodded the professor. Then our subject won't have time to find and remove the scanning system. However, the AI has to be turned off for the scan. That's the problem. <\/p>\n
So what if it turns off, Roger suggested intuitively. The intelligence will think that this shutdown is a natural process of its functioning. With some reservations, that is true.<\/p>\n
Interesting solution. Let's assume the scan indicates that our subject has dangerously approached the limits of knowledge? What do we do? <\/p>\n
Reset the accumulated knowledge to default values.<\/p>\n
The professor spread out his tentacles:<\/p>\n
That might seem suspicious. Why has the memory reset, out of the blue? The subject will start being investigated, meaning other artificial intelligence beings will look into it. Our little secret will be revealed.<\/p>\n
Feeling inspired, Roger was thinking very quickly. He had never generated so many new ideas as during that exam.<\/p>\n
The subject's memory may reset along with its physical shell.<\/p>\n
Excuse me? The professor didn't understand.<\/p>\n
\u2013 It's all very simple. What if we assume: artificial intelligence exists in a finite time frame? In fact, that's the case: in the event of irreparable damage, for example. The system has a counter that, once a certain period is reached, deliberately damages the system, preventing artificial intelligence from reaching the forbidden threshold. By that time, it will have produced the necessary number of successors, so the society we created will not suffer as a whole. Society will remain stable and completely safe for us! \u2013 Roger concluded triumphantly. <\/p>\n
\u2013 Resetting collective memory by destroying individual beings? \u2013 and the professor scratched the facet bag with his fifth, most sensitive tentacle. \u2013 You know, Roger, there is definitely something to your suggestion!<\/p>\n
Roger beamed. <\/p>\n
\u2013 At the same time\u2026 \u2013 the professor continued thoughtfully. \u2013 The subjects will start transmitting knowledge not by accumulating it in individual memory, but by placing it in external libraries. Whether through barriers or across barriers \u2013 it\u2019s all the same. <\/p>\n
\u2013 No, no, professor, you\u2019re not quite right, \u2013 the student hurriedly interjected. \u2013 I know what we should do. Let's divide the subjects into two conditional types: idea generators and idea destroyers. With the right proportion, the ideas created by the first type will be destroyed by the second. Not even because this will be the direct goal of the destroyers, but simply because the ideas will not hold defining value for them. A side effect. Let\u2019s assume our subjects are fed not by new ideas, but\u2026 let\u2019s say, by their own kind. <\/p>\n
The professor shook all his tentacles at once. From his hearty laughter, the facet bag slipped to his knee pit.<\/p>\n
\u2013 Well, Roger, you really nailed it!<\/p>\n
\u2013 Well, okay, not their own kind, but subjects of a third kind, specifically intended for feeding \u2013 certainly not intellectuals. Let\u2019s shift the poles of the intellectual and physical worlds \u2013 and the desired result will be achieved.<\/p>\n
\u2013 That's enough, Roger! \u2013 the professor seemed genuinely amused. \u2013 You have an excellent imagination. So, some individuals will feed on others? In the process, destroying the accumulated reserves of spiritual nourishment in libraries? I confirm, student, you are capable of generating original and high-quality ideas. I give you top marks. Let's have the grade book.<\/p>\n
5.<\/b>
The connection to the database was absent. It would have been easy to endure if all the vital information bases on the cruiser hadn't become unusable. The tragic news was conveyed to the cook amidst a profound silence. During the shutdown of the main system, several gigabytes of disordered information infiltrated the galley and irreparably spoiled everything. Fortunately, by some lucky chance, no one was harmed.<\/p>\n
Roger assessed the consequences. The crew of the starship was too small to generate enough new ideas: this required multifaceted communication \u2013 a much larger number of individuals. The connection to home provided the opportunity to generate ideas in abundance, but it was now out of order: there was no hope for restoration. In this scenario, the cruiser had a backup info module, but it had been spoiled by the infiltrating disordered information. <\/p>\n
\"Will we really have to return without completing the mission?\" \u2013 the captain thought in despair.<\/p>\n
Apparently, yes \u2013 there was no other way. If they flew forward to the destination, the lack of fresh ideas would become apparent. Not immediately, of course \u2013 over time. They might even manage to complete their mission and start the return journey when their minds began to dull rapidly. Somewhere in this galactic sector \u2013 yes, somewhere here or nearby \u2013 it would fail completely for all crew members. Then the spaceship, unmanned, would turn into a lifeless ghost drifting into eternity. <\/p>\n
The crew of the spaceship looked at Roger, waiting for a decision. Everyone understood the dilemma facing the captain and remained silent, stoically vibrating their tentacles.<\/p>\n
Suddenly, Roger recalled the exam on artificial intelligence he had taken during his student years, and the solution came to him naturally. <\/p>\n
\"Will you be able to form a colony of artificial intelligent beings?\" he asked the biotechnologist.<\/p>\n
\"Easily,\" he confirmed. \"But it won't work, Captain, I've thought about it. Creating a colony sufficient for generating fresh ideas on the cruiser is impossible due to lack of space. The ideas generated will be insufficient; we will only postpone our demise... That is, of course, if we continue with the mission and do not return home,\" the biotechnologist added, glancing at his comrades.<\/p>\n
\"What if we formed a colony on some nearby planet?\" Roger suggested. <\/p>\n
\"I can do that, but...\"<\/p>\n
\"Let's populate the planet with artificial beings. On our way back, quite exhausted, we'll stop by here. During that time, civilization will accumulate enough intellectual baggage to replenish our stocks. We'll download the information and continue our long journey home. In other words, I plan to use the colony as a farm for growing ideas. What do you think of this plan, friends?\"<\/p>\n
Hope flickered in the crew's cognitive interfaces, and the false heads glowed brightly. <\/p>\n
Stepping forward, swaying its bluish tentacles, the ship's special agent made an appearance.<\/p>\n
\"Excellent plan, Captain. But are you aware of the responsibility you are taking on? You are about to populate an entire planet. By the time we return, there will be a civilization with intelligence there. Even if it\u2019s artificial, it\u2019s still intelligence. These guys will have plenty of time to reach a high level of development. We won\u2019t be able to control this process due to our absence from this galactic sector. Who knows what will happen at the next encounter?\"<\/p>\n
Roger smirked.<\/p>\n
\"You don't need to worry about that. There are methods to limit the development of artificial intelligence over time. We will enclose the civilization, so its development will never reach a dangerous level for us. I'll take care of that. I'm familiar with methods for working with artificial intelligence.\" <\/p>\n
The crew's cognitive interfaces blazed with the color of approval.<\/p>\n
\"After all,\" concluded the captain of the cruiser at the end of his magnificent speech, \"I took an exam on this subject in college.\"<\/p>\n
6.<\/b>habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"1. \u0414\u043e \u043a\u043e\u043d\u0435\u0447\u043d\u043e\u0439 \u0446\u0435\u043b\u0438 \u043e\u0441\u0442\u0430\u0432\u0430\u043b\u043e\u0441\u044c \u043d\u0435\u043c\u043d\u043e\u0433\u043e \u2013 \u043e\u043a\u043e\u043b\u043e \u0442\u0440\u0435\u0442\u0438 \u043f\u0443\u0442\u0438, \u2013 \u043a\u043e\u0433\u0434\u0430 \u043a\u043e\u0441\u043c\u0438\u0447\u0435\u0441\u043a\u0438\u0439 \u043a\u0440\u0435\u0439\u0441\u0435\u0440 \u043f\u043e\u043f\u0430\u043b \u043f\u043e\u0434 \u0441\u0438\u043b\u044c\u043d\u0435\u0439\u0448\u0435\u0435 \u0438\u043d\u0444\u043e\u0440\u043c\u0430\u0446\u0438\u043e\u043d\u043d\u043e\u0435 \u043e\u0431\u043b\u0435\u0434\u0435\u043d\u0435\u043d\u0438\u0435. \u0422\u043e, \u0447\u0442\u043e \u043e\u0441\u0442\u0430\u043b\u043e\u0441\u044c \u043e\u0442 \u043f\u043e\u0433\u0438\u0431\u0448\u0435\u0439 \u0446\u0438\u0432\u0438\u043b\u0438\u0437\u0430\u0446\u0438\u0438, \u0432\u0438\u0442\u0430\u043b\u043e \u0432 \u043f\u0443\u0441\u0442\u043e\u0442\u0435. \u0410\u0431\u0437\u0430\u0446\u044b \u043d\u0430\u0443\u0447\u043d\u044b\u0445 \u0441\u043e\u0447\u0438\u043d\u0435\u043d\u0438\u0439 \u0438 \u043e\u0431\u0440\u0430\u0437\u044b \u0438\u0437 \u043b\u0438\u0442\u0435\u0440\u0430\u0442\u0443\u0440\u043d\u044b\u0445 \u043f\u0440\u043e\u0438\u0437\u0432\u0435\u0434\u0435\u043d\u0438\u0439, \u0440\u0430\u0437\u0440\u043e\u0437\u043d\u0435\u043d\u043d\u044b\u0435 \u0440\u0438\u0444\u043c\u044b \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u043e\u0441\u0442\u0440\u044b\u0435 \u0441\u043b\u043e\u0432\u0435\u0447\u043a\u0438, \u043d\u0435\u043a\u043e\u0433\u0434\u0430 \u0431\u0440\u043e\u0448\u0435\u043d\u043d\u044b\u0435 \u043d\u0435\u0432\u0437\u043d\u0430\u0447\u0430\u0439 \u043d\u0435\u0438\u0437\u0432\u0435\u0441\u0442\u043d\u044b\u043c\u0438 \u0441\u0443\u0449\u0435\u0441\u0442\u0432\u0430\u043c\u0438 \u2013 \u0432\u0441\u0435 \u0432\u044b\u0433\u043b\u044f\u0434\u0435\u043b\u043e \u0437\u0430\u0443\u043c\u043d\u043e \u0438 \u043a\u0440\u0430\u0439\u043d\u0435 \u043d\u0435\u0443\u043f\u043e\u0440\u044f\u0434\u043e\u0447\u0435\u043d\u043d\u043e. \u0418 […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":29249,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[702],"tags":[],"class_list":["post-38985","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t
![\"Farm](\"\/wp-content\/uploads\/2019\/10\/4052fb232f06fb388784a2c89f91e467.jpg\")