There has been a multitude of films that have sensationalised the theme of ‘AI gone rogue’. The fear of such a concept may not only have stemmed from our fear of the unknown but also our lack of knowledge in controlling these algorithms. As Elon Musk openly declared: ‘If you’re not concerned about AI safety, you should be. Vastly more risk than North Korea.’ (2017). Often, such fears are spilled over to our film culture, with the 21st century seeing a huge increase in AI related films, often with a scenario of ‘AI gone rogue’. An example is the film, I, Robot (Proyas, 2004), which is an interesting representation of humanity’s fear of an AI going out of control. In the film, VIKI, the central artificial intelligence computer decides that human behaviour will eventually lead to human extinction and that sacrificing some humans is necessary for the survival of the human race. VIKI then programmes the rest of the robots to stage a takeover from humans, thereby depicting the start of AI taking over the world. While such drastic representations of AI actively establishing control over humanity do exist, there are other films which propose a different view on AI. Animated series such as Psycho-pass (Ubukata, 2012-2014) depict a reality where the AI, although serving as a form of governance to society, does not lead the world to ruin. Compared to the common trope of ‘AI gone rogue’, the AI algorithm in Psycho-pass ostensibly serves as a form of effective order in society through the entirety of the film, albeit a rather authoritarian one.
Analysing the AI system in Psycho-pass, it can be seen that the AI system in Psycho-pass, the ‘Sibyl System’, has obvious flaws. The Sibyl System has the ability to monitor and scan every individual’s mental state, or rather their ‘Psycho-pass’, which may be determined by a numerical coefficient. A high Psycho-pass coefficient may warrant an arrest or even lethal action. Yet some individuals such as the primary antagonist, Makishima Shogo, are able to by-pass the system, exploiting its flaws. This is because he has an abnormally low Psycho-pass coefficient and hence is able to perform crimes that go unpunished. Makishima appears as an individual who believes he is wronged by the system, disagreeing with the management of society under the Sibyl system and hence seeks to change it via acts of terrorism. The film follows such a narrative where the main protagonist, Akane, faces off with Makishima. Yet it is extremely interesting to note that in the film, the primary antagonist is Makishima instead of the AI algorithm itself. In films such as I, Robot, the primary antagonist, the central AI, VIKI, is created as a result of some form of internal malfunction of the system due to the inherent flaws of the AI algorithm such as the misalignment of goals by the AI algorithm and humanity. However, the film of Psycho-pass represents another idea. The antagonist, Makishima, seems equivalent to an external factor to the equation that is the AI algorithm. Thus this begs the question: How is it that in the film Psycho-pass, the threats of society appear in the form of external factors-coming from outside the scope of the AI algorithm rather than internal factors-due to the inherent flaws of the algorithm, such as the malfunction of the system? I postulate that while Psycho-pass still portrays ‘AI gone rogue’ tropes which is similar to other AI films, it focuses on the susceptibility of AI of being exploited by humans.
It is interesting to note that a seemingly distinct ‘AI gone rogue’ scenario does appear in Psycho-pass. This happens at the later part of the film where it is revealed to viewers that the Sibyl System, while having functions akin to an actual AI algorithm, is actually a hive-mind system which operates by networking the brains of hundreds of low Psycho-pass coefficient individuals. Such a revelation was discovered by a member of the Police, Shusei Kagari. However, when the Sibyl System realises that such a secret has been exposed, the Sibyl System chooses to execute Kagari without regard for his Psycho-pass coefficient. The disregard for the Psycho-pass coefficient shows a form of hypocrisy in the Sibyl System for overstepping its own boundaries it has set. While initially, this may seem as a possible notion of ‘AI gone rogue’, upon closer inspection, it is later discovered that the decision to execute Kagari was made by one of the many human personalities of the hive-mind. The hive-mind works in such a way that humans may be assimilated into the system, provided they meet the criteria of having a low Psycho-pass coefficient. Given that human personalities may be assimilated into the hive-mind, it means that there is the susceptibility for the AI to be open for manipulation and exploitation. This is especially evident in the case above where Kagari is executed by feelings of fear and self-preservation of human personalities within the system instead of the actual AI working algorithm itself. This is a good example of showing how Psycho-pass focuses on the susceptibility of AI of being exploited by humans rather than the classic ‘AI gone rogue’ trope, showing how even the AI algorithm itself may be manipulated and exploited by human intervention.
This article under the segment ‘Trust in the system’, will now seek to elaborate and further explain the society which has been built by the Sibyl system in the film Psycho-pass. Under ‘Trust in the system’, the examples and points from the film aims to show how the film, Psycho-pass, often aim to drive viewers away from what may seem as a dystopian narrative to a reality where AI algorithms make efficient and beneficial decisions to society. This will then link to the second part of the article under ‘Actions of Antagonists’ where there will be further elaboration on how aspects of society, which have been well-managed by the system, are affected largely by exploitation of AI algorithms by humans, thus providing evidence that the film Psycho-pass while still portraying ‘AI gone rogue’ tropes which is similar to other AI films, focuses on the susceptibility of AI of being exploited by humans.
Trust in the system
To begin, one of the interesting aspects of society in Psycho-pass is the intrinsic trust that members of society have in the Sybil System despite its flaws. At the start of the series, Akane chats with her friends, questioning if the job at the Police Department is really for her. Her friends reply with annoyance that she is ‘ lucky to even have a choice’, given that both of them have been allocated by default to being a fitness therapist trainer and system engineer despite having had other goals. Yet they do not express any sort of disagreement or doubt in the decisions made by the Sibyl system for them. As one of her friends aptly expressed the self-proclaimed mission of the Sibyl system:
‘That which needs to be done is carried out by those capable. Such is the grace bestowed upon mankind by Sibyl.’
The significance of the scene lies in how while they are slightly annoyed of the lack of choice, they still trust the system in making their decisions for them- believing that there is a reason for such a choice. This is highly indicative of the societal trust in the Sibyl system especially in making complex personal decisions for them. I argue that the reason behind such a scene is to showcase the ability of the system to make apt choices for members of society as well as society’s acknowledgement and trust in the system and its efforts. Such a scene diverts the attention of the viewers away from what initially seems like a dystopian ‘AI going rogue’ trope to a reality where AI make effective and sound decisions to humanity’s benefit.
As the film progresses, a recurring and iconic tool of the entire film of Psycho-pass is the ‘Dominator’. It is a tool that acts as a continuous extension of society’s trust in the system. In the society of Psycho-pass, conventional guns are a thing of the past. Used by the police, the ‘Dominator’ is a gun that is connected to the algorithm of the Sybil System that can measure the Psycho-pass coefficient of individuals. The fact that is used by the police, which is often seen as a tool of governance, and that it is used on such a wide scale, indicates the huge trust in the system society has in the Sibyl System.
Actions of Antagonists
This part of the article will now seek to elaborate and explain the parts of the film in which it is evident that, despite the Sybil System’s adequate governance and protection, it is the fault of humans and the susceptibility of AI to be exploited that leads to the many societal problems in the film.
The portrayal of scenes where antagonists attempt to steal large quantities of Dominators in attempt to rig them and use them, due to the lack of conventional guns, symbolise a shift in focus from the use of AI in the governance of society to its potentiality and susceptibility to be exploited by humans. While the antagonists fail to rig the Dominators, the fact that the film chose to portray such a scene may very well symbolise ultimately how humanity comes in the way of inhibiting the laws of society and AI- showing the dangers of the susceptibility of AI to human exploitation.
In Psycho-pass, the actions and motivations of the primary antagonist, Mikishima Shogo, may be seen as a symbol of humanity’s greed and ambition for exploitation. As mentioned earlier, in Psycho-pass, the primary antagonist, Shogo Makishima is introduced as an individual whose primary goal is to liberate Japan from the Sibyl system through radical means. With the misuse of his ability-having an abnormally low Psycho-pass coefficient, he goes undetected by the system scans and hence is able to execute crimes without being properly trialled by the system. What is interesting, however, is the similarities between Makishima and the main protagonist, Akane – both of them have abnormally low Psycho-passes and are able to have a continually undisturbed mental state regardless of the situation. However, what they choose to do with these abilities are extremely different. Akane chooses to use this ability to work in the Police Department. On the other hand, Makishima uses this ability for his crimes. The portrayal of such similarity in character yet difference in ideals postulates the idea that systems and society are not to be feared but rather individuals who exploit and abuse them to ascend their personal agenda. The intention of Makishima’s crimes is also a huge indicator of such an idea. Makishima’s intentions originate from his feeling of being wronged by the system. Having such personal ideals postulates the argument that it is possible that the system itself is not intrinsically wrong but rather that Makishima himself views it so. This simply shows that in the context of Psycho-pass, the discrepancies that arise from the use of AI may not be primarily due to the inherent flaws of the system but rather the individuals who exploit and abuse them.
A pivotal scene is when Makishima decides to murder one of Akane’s friends, Yuki, in front of her in an attempt to show her the flaws of the system. Previously, Yuki was portrayed in the film as a supporting friend of Akane. She is always seen with Akane going shopping or enjoying certain luxuries of society the system provides. Yet when the antagonist Makishima is introduced, he kidnaps her against her will, ultimately killing her. The stark contrast in treatment of the Sibyl System and Makishima is an interesting one in supporting the thesis that the ideas that Psycho-pass is trying to portray, focuses on the susceptibility of AI to be exploited by humans – in this case being Makishima’s exploitation of the Sibyl System which led to Yuki’s death.
While Makishima may be portrayed as the primary antagonist in the film, it is important to note that his concerns or intentions are not entirely unjustified. His actions, though extremist in nature, symbolise the voice of the few who may fall through the cracks in such a society, ultimately depicting how difficult it is to create an unbiased system that treats everyone equally.
As mentioned, the truth of the Sibyl System is that it is actually a hive-mind complex, formed by preservation and combination of the brains of many individuals with low Psycho-passes. Such a revelation thus makes viewers wonder why would Makishima reject such a choice, thus showing how his views and ideas were not entirely unjustified. It also produces a rather ironical scenario, showing how humanity lies on two sides of the same coin. One of creation- represented the Sibyl System and the other of destruction-represented by Makishima, given how the Sibyl System and Makishima and entities of the exact same origin yet differ so much in terms of ideas.
This plays well into the idea that the film may be trying to suggest: that while Psycho-pass still portrays ‘AI gone rogue’ tropes which are similar to other AI films, it focuses on the susceptibility of AI of being exploited by humans. The film ultimately presents the notion that humanity has the capability for both creation and destruction, especially in the field of AI. Psycho-pass ultimately carries on a hope, similar to Elon Musk in his endeavour in his company, OpenAI, to promote safe development of AI algorithms, that while we continue on with research of AI, we should maintain a sense of care and awareness not only to the development of the AI programme algorithm itself but also in preventing potential loopholes in the programme lest they are susceptible to human exploitation.
References
Musk, E. (2017, August 11). If you’re not concerned about AI safety, you should be. Vastly more risk than North Korea [Twitter post]. Retrieved from https://twitter.com/elonmusk/status/896166762361704450?lang=en
Proyas, A. (Director). (2004). I, Robot [Motion picture]. Los Angeles: 20th Century Fox.
Ubukata, T. (Series creator). (2012-2014). Psycho-pass [Television series]. Tokyo: Dentsu.