Regulating AI in the Finance Sector in India Jai Vipra February 2020

IV Preface “After all, we make ourselves according to the ideas we have of our possibilities.” V.S. Naipaul There is no doubt that the technological advancement has become the game changer of our times. From the Industry 4.0 discourse launched in Germany in 2011 to the scientific advisory report presented to the former US president Barrack Obama on big data and privacy concerns in 2014, to India’s NITI Aayog Artificial Intelligence for All strategy of 2018. A lot of debates have culminated in the questions about the Future of Work in the context of the International Labour Organisation’s Centenary in 2019. Triggered by the disruptive forces of technology based start-ups and new business models, a new race for innovations and war for talents has arisen and with it, a new form of global and fierce competition. Technology has become the holy grail of progress though it did not take long to realise that there is a social dimension attached to it. The platform economy has had severe effects on the bargaining power of suppliers and workers. Data analytics opened a whole array of ethical questions regarding personal tracking and privacy. Further, technological upgrades create productivity gains by efficiency which in turn requires reduced human labour. This poses a particular threat to emerging economies, like India, which need to create new jobs on massive scale for its young and growing population. The utopia around Artificial Intelligence in the times of jobless growth presents a whole new set of challenges. Is the Indian economy ready to ride the AI wave? Who will benefit from AI: investors, big tech, users, or society as a whole? What is and can be India’s role in this global race for innovation? Is tech gender neutral? What about privacy and user protection? How to ensure decent work and social protection in this new age tech revolution? But mostly, how can we turn AI FOR ALL into a reality? To foster this debate, the FES India Office has teamed up with several experts and organisations across the country to explore ground realities with the objective to understand how technology is already unfolding in selected sectors, draft scenarios of what might happen and to ensure proper safeguards are put in place at the right time. Artificial Intelligence like any other technology is neither good nor bad. It is what we make out of it- the rules and regulations – which define the outcome of the game. Just like other countries, in India too, a mass scale application of AI is far from being established. It is still in a nascent phase and can be moulded into a success story. A success story in India AND an Indian success story for all. Patrick Ruether and Mandvi Kulshreshtha February 2020 Friedrich-Ebert-Stiftung, New Delhi Regulating AI in the Finance Sector in India

V Note of thanks Friedrich-Ebert-Stiftung(FES) India office is thankful to its partner IT for Change(ITfC) for preparing this research paper. IT for Change aims for a society in which digital technologies contribute to human rights, social justice and equity. ITfC is preparing a series of research papers for FES tracing AI’s enigma in sectors such as agriculture, ports, education and fintech. We are grateful to our colleagues at ITfC for preparing the research, drafting this paper and refining the manuscript to reflect our joint vision. We have to express our appreciation to the Editorial team at ITfC, for their constructive contribution and valuable time during the course of this research. Regulating AI in the Finance Sector in India

VI Contents List of abbreviations....................................................................................................................................... VII Acknowledgements...................................................................................................................................... VIII 1. Overview...................................................................................................................................................1 2. Main trends in the use of AI in fintech in the Indian context................................................................. 2 3. Risks posed by AI in fintech...................................................................................................................... 3 4. Taking stock of the existing regulatory response in India...................................................................... 6 5. What does the regulatory response need to be?....................................................................................7 i. Data ownership and governance......................................................................................................7 ii. Governance of algorithms...............................................................................................................8 iii. Regulatory techonology or RegTech................................................................................................8 6. Conclusion............................................................................................................................................... 10 Endnotes....................................................................................................................................................... 11 Regulating AI in the Finance Sector in India

VII List of abbreviations AI EASE IoT IRDAI KPMG MFI MSME NASSCOM Artificial Intelligence Emotional Analytics for Social Enterprises Internet of Things Insurance Regulatory and Development Authority Klynveld Peat Marwick Goerdeler Microfinance Institution Micro, Small and Medium Enterprises National Association of Software and Services Companies NDSAP OECD RBI SEBI TreDS USD National Data Sharing and Accessibility Policy Organisation for Economic Co-operation and Development Reserve Bank of India Securities and Exchange Board of India Trade Receivables Discounting System United States Dollars Regulating AI in the Finance Sector in India

VIII Acknowledgements This research report was commissioned by FriedrichEbert-Stiftung- India Office. Author: Jai Vipra Peer Review: Anita Gurumurthy and Nandini Chami Regulating AI in the Finance Sector in India

Overview 1 1. Overview Financial products and services have perhaps seen the quickest adoption of digital technology among all sectors in India. Regulation in this sector has also traditionally been nimble, if sometimes controversial. The use of Artificial Intelligence(AI) in fintech--- like in any sector based on digital technology--- is also rising. The report of the Reserve Bank of India’s(RBI) Working Group on FinTech and Digital Banking has categorised fintech innovations into: payments, clearing and settlement; deposits, Recent projections from lending and capital raising; 2018 suggest that fintech market provisioning; is likely to add USD 700 investment management; billion to the country’s and data analytics and risk GDP by 2025. management. 1 In 2016, NASSCOM and KPMG estimated the value of the Indian fintech market to be USD 1.2 billion, predicting that it would increase to USD 2.4 billion by 2020. 2 More recent projections from 2018 suggest that fintech is likely to add USD 700 billion to the country’s GDP by 2025. 3 As expected, due to differences in digital literacy levels and quality of connectivity infrastructure, fintech has seen greater adoption in urban areas. 4 In keeping with trends worldwide, the Indian ecosystem of fintech innovation includes fintech startups as well as technological innovation within existing Traditional financial financial service companies. institutions and Traditional financial companies are partnering institutions and companies with fintech startups as are also partnering with part of their business fintech startups as part of modernisation strategies. their business modernisation strategies. 5 These new forms of collaborations, however, do not preclude structural change in financial service markets. For example, many analysts predict that banking could change into a platform service, on top of which a variety of other products and services can be built, leveraging the customer data that is generated through digital transactions. 6 This paper analyses key trends in the deployment of AI in the fintech sector in India. First, it discusses the predominant uses of AI in Indian fintech. Then, it moves on to analysing the potential risks of the use of AI in financial products and services, particularly focusing on the exclusion and concentration of economic power that it can trigger. It then moves on to take stock of the adequacy of the existing regulatory framework and concludes by offering a set of recommendations to strengthen the same. Regulating AI in the Finance Sector in India

2 Main trends in the use of AI in fintech in the Indian context 2. Main trends in the use of AI in fintech in the Indian context The report of the RBI Working Group positions Artificial Intelligence, blockchain and Internet of Things(IoT) as the three technologies that will cause major changes in financial products and services in India. 7 Across industry reports, government documents and media reports, the use of AI in fintech in India seems to revolve around the following(non-exhaustive) list of considerations: (a) Increasing coverage of credit services: India has a large gap in the demand for, and supply of, credit. People who want to borrow are unable to do so from institutional lenders, because they cannot provide collateral and do not have a documented Fintech, often using AI, credit history. Credit scoring is able to use alternate is meant to reduce the need data sources to create for collateral, as it may help customer profiles predict default. However, and predict rates of according to one estimate, default, thereby helping the fact that traditional targeting. credit scoring uses only loan repayment data leaves 90 per cent creditworthy people out of the credit net. 8 Fintech, often using AI, is able to use alternate data sources to create customer profiles and predict rates of default, thereby helping targeting. Some alternate data sources used are: social media data, payment behaviour on apps and utilities, call logs, location etc. 9 AI driven assessment of creditworthiness is also carried out by peer-to-peer lending platforms. 10 (b) Reducing costs: Automation through AI is also helping reduce costs in fintech operations. For example, banks are now partnering with fintech startups to predict farmer behaviour, which reduces the costs of going to Automation through AI farms to assess farmland. 11 is helping reduce costs in Financial service firms now fintech operations. also use AI-enabled chatbots to interact with customers, reducing the need to hire personnel for this purpose. 12 AI-enabled multilingual chatbots are now being developed for use in rural India. 13 Another example is the use of AI for the backend, including for automation of report writing, accounting, and data entry. 14 (c) Countering cybersecurity threats: As they continue to digitise, financial institutions are also using AI to counter increasing cybersecurity threats, either by developing these solutions in-house or outsourcing their development. 15 A survey of IT security and IT professionals in India found that 24 per cent of them use some form of machine learning, and most believe that AI will help improve cyber security. 16 (d) Facilitating regulatory compliance: AI can help detect anomalous transactions and flag them for review, thus enabling financial institutions to comply with laws and regulations such as those on anti money-laundering. 17 IBM Watson Financial Services, for example, provides AI technology for complying with changed regulations, Know Your Customer laws and so on. 18 Regulating AI in the Finance Sector in India

Risks posed by AI in fintech 3 3. Risks posed by AI in fintech In the financial sector, the systemic effects of individual actions can be severe, due to interlinkages and the scale of the system. This fact is the basis of most macroprudential regulation. The introduction of AI into financial decision-making can exacerbate the risk of such systemwide repercussions, given the rapidity of automated decision-making and difficulties in arresting or reversing its impact. The particular implications of the use of AI in finance are discussed below: (a) Vicious cycles of financial exclusion: The use of demographic data in credit rating algorithms can, in the first instance and depending on the predictive model used, exclude certain The use of demographic communities from credit, data in credit rating and this exclusion and the algorithms can exclude resultant deprivation can certain communities from further nudge algorithms credit, and this exclusion to exclude them, creating and the resultant a cycle of exclusion. This deprivation can further concern is separate from nudge algorithms to the concern of algorithmic exclude them, creating a fairness and bias, but it is cycle of exclusion. related. O’Dwyer(2018) points out how this form of exclusion-by-geography mirrors what used to happen in the US in the 1930s, where entire neighbourhoods were coded by creditworthiness, which further perpetuated exclusion and poverty. Anti-discrimination laws changed these practices. The decision to provide credit or not gets transferred to the algorithm and in the absence of regulations specifically targeting algorithms, there is now a danger of the resurgence of practices described above. 19 While there is no systematic evidence of such exclusion in India yet, fintech startups are already creating alternate credit scores for a large section of the population. It is important for regulators to remain aware of the potential problem and prevent it. 20 (b) Pro-cyclicality: The use of automated investment advisory services in India is small but is projected to grow faster than the global average. 21 Automated advisors can execute millions of trades for as many retail investors in a matter of seconds and exacerbate a downturn. This is the problem of pro-cyclicality. The Financial Stability Board has pointed out that automated advisors could exhibit greater herding behaviour--- a tendency to follow the crowd even if the decision is irrational –- than human advisors, especially if similar risk models are used by many Automated advisors actors. 22 Besides, automated can execute millions of advisors are aimed at low trades for as many retail and middle income investors investors in a matter of as they are low cost, which seconds and exacerbate a creates a situation of downturn. disproportionate harm when the algorithm malfunctions. 23 This issue, however, only arises if the algorithms and their uses have not accounted for these possibilities. Algorithms can be required to contain safeguards against pro-cyclicality, or regulators can prevent different risk models from being too similar. However, designing such safeguards is difficult as people behave differently with automated advisors and human advisors. For example, some analysts believe that when dealing with automated advisors, people are more likely to have knee-jerk reactions in bear markets. In the same circumstances, human advisors may be more successful in persuading their clients against such reactions. 24 (c) Unintended and conscious market collusion: OECD finds that algorithms enable certain conditions (such as speed of transactions) for collusion among firms in markets. It also finds that self-learning algorithms, such as those that constitute AI, can unconsciously and unintendedly achieve collusion. Essentially, a cooperative equilibrium can be reached as algorithms quickly learn from In financial service and react to competitors’ markets that are already actions in markets that heavily digitalised are prone to collusion. 25 In and automated, the financial service markets introduction of AI can that are already heavily create a vulnerability to digitalised and automated, collusion. the introduction of AI can create a vulnerability to collusion. Of course, intentional collusion through algorithms may also occur. Ezrachi and Stucke(2017) argue that there are three market conditions necessary for conscious algorithmic collusion to arise: Regulating AI in the Finance Sector in India

4 Risks posed by AI in fintech markets have to be concentrated with homogenous products, a credible deterrence has to exist to prevent deviation from optimal behaviour(of collusion) and buyer power has to be limited. 26 Financial markets often fulfil these conditions--- for example, when insurance markets have many similar products across brands--and regulators need to stay alert to anti-competitive outcomes arising from the use of AI. (d) Reinforcement of Big Tech domination: Parallel to the development of fintech, or the use of new technology in providing financial products and services, has been the development of techfin, or the provision of financial products and services by The entry of technology technology companies. The and e-commerce entry of technology and companies into the e-commerce companies into financial sector has been the financial sector has been called the“Uber moment called the“Uber moment of finance”. of finance”. 27 However, financial services have been crucial to the rise of transnational digital companies for some time now. Alipay, a payments system, helped Alibaba gain market share over its competitors early on. Today, Alipay has been transformed into Ant Financial and offers not only a payments system with 870 million users, but also a money market fund and a credit platform. 28 Similarly, Amazon offers lending and payments, and Google, Microsoft, Samsung, WhatsApp and Apple all offer payments systems or integration with payments systems. Sometimes the penetration of technology giants into the financial sector is not obvious: for instance, the voice of HDFC Bank’s chatbot Eva is powered by Amazon’s Alexa. 29 This would, depending on the data sharing arrangement between the involved parties, provide Amazon valuable data on financial transactions. This raises data capture concerns as Amazon is already a data giant. It can easily dominate a sector on the basis of its already existing data, which can allow it to build superior services, such as loans that are targeted to the correct demographic. Libra, the latest development in techfin, has its own set of issues that are beyond the scope of this paper. Some analysts believe that big tech companies’“massive customer bases and vast cash reserves” will help them scale faster than other fintech firms. 30 Others expect that technology companies’ agility, depth of data on preferences and behaviour, and user-facing nature allow them to make more nuanced analysis than companies that are financial companies first and technology companies second. 31 The likely use of AI in the provision of financial products and services by Big Tech throws up distinct issues for analysis. The use of AI and privately-held big data by technology giants to provide financial products The use of AI and and services, can create privately-held big data competition issues in the by technology giants financial sector, given that the to provide financial quality of algorithms is partly products and services, a function of the underlying can create competition data. Such use can also issues in the financial create(further) competition sector. issues in the technology or e-commerce sectors themselves, as technology and e-commerce companies use(privately held) data from their own financial services to increase efficiency in their business practices. The lack of close coordination between financial regulators, technology regulators and e-commerce regulators will end up being a condition that creates policy gaps as techfin causes these sectors to merge. (e) Algorithmic control of behaviour: We have already seen that disparate datasets are used for making lending and insurance decisions by fintech providers in India. The following datasets are already in use: location data is used to verify whether a person works where they claim they work; food delivery and ride hailing app data is used for “behavioural scoring” and to analyse patterns of usage; e-commerce data for how often a customer returns products; mobile data for browsing history and whether it conforms to the customer’s social and professional profile; video and audio data to recognise emotions and determine whether a borrower is being coerced(for example, through an application called Emotional Analytics for Social Enterprises (EASE)); customer facial image data in bank branches to determine service quality, etc. 32 These technologies are not merely targeted at the urban elite; small business owners in Indian small towns are frequently the target group. 33 Regulating AI in the Finance Sector in India

Risks posed by AI in fintech 5 When people know that such different datasets are being used to make decisions as With the use of big data, important as the sanctioning behaviours that have not of a loan or the determination been linked till today are of an insurance premium, being linked, and there they naturally attempt to is no exact precedence to manipulate this data through this regulatory challenge. their behaviour. Sometimes this is even the goal, such as in a health insurance scheme that would prefer its beneficiaries to eat healthy. However, with the use of big data, behaviours that have not been linked till today are being linked, and there is no exact precedence to this regulatory challenge. Selbst et. al.(2018) call the process of the introduction of technology into a social system and the consequent change in existing behaviours and values, the“ripple effect”, and the failure to account for this process while designing technology, the“ripple effect trap.” The algorithm begins to control behaviour in distortionary ways, for example, by privileging quantifiable metrics over qualitative metrics. Certain social values are Related to the problem easier to quantify than of algorithmic control of others, and there is a risk that behaviour is also a lack when the decision is made of control brought in algorithmically, these values by algorithms, in cases get privileged just by virtue of where it is unclear to the their quantifiability. 34 Related subject what behaviour to the problem of algorithmic is rewarded and what is control of behaviour is also punished. a lack of control brought in by algorithms, in cases where it is unclear to the subject what behaviour is rewarded and what is punished. Regulating AI in the Finance Sector in India

6 Taking stock of the existing regulatory response in India 4. Taking stock of the existing regulatory response in India Regulators in India are taking some gradual steps to grapple with the challenges of fintech. There are regulations around specific fintech products and services, such as peer-to-peer lending(RBI), digital payments (RBI), automated advisors(SEBI), wearable technology for insurance(IRDAI) working group report), 35 virtual currency(Ministry of Finance committee report) etc. The RBI is also set to introduce a regulatory sandbox for fintech. 36 Examples of major regulatory developments in response to the changing nature of the finance sector in India include RBI’s framework for licensing of payments banks, and banks being allowed to use the e-Know Your Customer(e-KYC) process for opening accounts. Government infrastructure has also allowed and shaped the development of fintech. The Pradhan Mantri Jan Dhan Yojana(financial inclusion program of Government of India) created millions of bank accounts, Aadhaar (unique identification number) and IndiaStack(a unified software platform) were launched, and are being used as infrastructure for authentication, payments, document verification, and so on. The Trade Receivable Discounting System(TreDS) was launched to help MSMEs discount their trade receivables and raise funds. The National Data Sharing and Accessibility Policy(NDSAP) set up an It is unclear how Open Data Portal to share existing regulations government data. apply to fintech service providers, and that Saman et. al.(2018) provide fintech companies often a comprehensive overview challenge the premises of the existing regulatory of existing regulations, framework, its history and including those related how it relates to AI in the to privacy, security and Indian financial sector. 37 They liability. find that it is unclear how existing regulations apply to fintech service providers, and that fintech companies often challenge the premises of existing regulations, including those related to privacy, security and liability. Regulating AI in the Finance Sector in India

What does the regulatory responses need to be? 7 5. What does the regulatory response need to be? Regulation of AI in fintech so far has been mushrooming sector-wise, such as in payments, lending and insurance. This way of regulating nascent industries can be more appropriate than blanket regulations before the implications of a technology The regulation of AI become clear. However, such must be based on cross- fractured regulation can also sectoral, clear premises serve to create loopholes, that the government encourage gaming and articulates through enable forum shopping, that policy. is, companies choosing to classify their activities under the most favourably regulated sector. The regulation of AI must be based on cross-sectoral, clear premises that the government articulates through policy. To this end, the government and regulators need to make choices on a few fundamental aspects of AI in finance: i. Data ownership and governance: As already discussed, the use of big data and AI in finance brings enormous efficiency gains and also facilitates financial inclusion. For these reasons, the use of these technologies will only continue to rise, and banning them, or heavily limiting their use, would mean undoing all their benefits. At the same time, the problems caused or likely to be caused by AI are non-trivial. One attempt to solve these problems has been through instituting personal data protection measures. These measures seek to empower an individual to consent or not to consent to data use in the market. They also disallow the commercial use of certain categories of data. An example of the consent thinking turned into fintech Legal requirements policy is the RBI’s approval to prevent mixing of to Account Aggregators, personal data need to be which are entities that strictly enforced. share data from people to financial companies, while maintaining a record of consent. At one level, we need the consent framework. All manner of personal data can be used by technology companies, especially social media platforms, to offer or deny financial services. This becomes an issue if financial services get monopolised by these companies, and also because there are privacy concerns with using personal data for this purpose. Legal requirements to prevent such mixing of data need to be strictly enforced. At another level, there are many well-known issues with the consent framework: it unduly burdens the consumer, often markets do not provide any meaningful choice, and the externalities of individual consent are not taken into account. The concept of group privacy accounts for these externalities: some data, even when about an individual and freely shared by him/her, has implications on the privacy and well-being of an entire group. Further, the distinction between personal and non-personal data that this framework tries to establish is also unhelpful in many situations. Non-personal, nonsensitive data, when clubbed with other such data, can become personal and sensitive, or at least, its use can have detrimental effects that have nothing to do with whether the data is personal or not. The personal data protection framework is then inadequate by itself to fix the issues that AI poses in finance. Issues arise from questions that are much more fundamental: who is able to capture the value derived from a new technology? What is the nature of this value capture, and what impact does it have on the people as a whole? What mechanisms exist, or can exist, to operationalise this value capture? The value of AI can be secured for the public if the underlying resource, data, is owned by the people. The personal data protection framework is not able to answer these fundamental The value of AI can be questions because it secured for the public if skirts the issue of data the underlying resource, ownership. Countries data, is owned by the need to think about people. models of community data ownership that allow people to extract value from data and AI in ways that are beneficial to them. For example, if communities own their data, they Regulating AI in the Finance Sector in India

8 What does the regulatory responses need to be? can decide to share it only with insurance providers who do not use the data extractively. They can even create their own, publicly provided insurance products, as the derivation of value from data will no longer be the sole domain of the private sector. Such common ownership would have to be subject to democratic methods of control in order to ensure the realisation of the common interest. This is the question of data governance: what is the best way to govern this new, highly beneficial resource, in a way that ensures that everyone’s rights are protected? Some examples of public provision of data services already exist in Indian fintech. The RBI’s public credit registry is a recognition of the fact that infrastructural data services might be best provided publicly. 38 The data that the registry uses, its practices, organisational processes, impact, etc., can be open to public scrutiny. In the context of discontent with the way in which private credit rating agencies use data, there have been demands for the creation of such a registry in the US as well. 39 The registry is certainly open to abuse, but this is more a question of its appropriate governance than a contention about its existence. The question of monopolisation of financial services and products through the introduction of techfin also needs to be examined through this lens, as monopolisation arises partly from data control. ii. Governance of algorithms: AI algorithms might require regulation irrespective of the ownership of regulation around data. There are multiple ways to regulate algorithms: a. First, regulators can Transparency cannot be ask for algorithms to be accepted uncritically as a transparent. This can be regulatory tool. useful to prevent the use of inappropriate or unwanted data to provide a product. Transparency, of course, cannot be accepted uncritically as a regulatory tool. Transparency is frequently impossible, can drown one in too much information, reveal trade secrets, assign agency where there is none and compromise privacy, particularly when sensitive financial data is involved. 40 b. Second, regulators can require that the outputs of algorithms correspond with certain minimum standards. These standards can be those of nondiscrimination, or regulators can require that the outputs of an algorithm be better than a randomly generated set of outputs. Sometimes, however, these choices are political. People’s tolerance for levels of discrimination and quality of output change through time and material conditions. If this decision is made by regulators, there is a danger of mission creep and bureaucratisation of a political process. Regulatory agencies must recognise this and leave such decisions to elected legislative bodies. c. Third, regulations can require that the development and use of algorithms follow a certain process. Regulations on automated financial advisory are of this nature: the responsibility for algorithmic input has to be clearly delineated within the organisation, certain questions have to be asked and incorporated into the algorithm before advice is given, etc. As Regulators need to be another example, the careful about turning IRDAI Working Group on people into“moral InsurTech recommends crumple zones”, whereby that the misuse of data humans unfairly bear from wearables should liability for automated be tackled by“robust decisions, as no one can internal processes” be found to blame. followed by insurers. 41 Here regulators need to be careful about turning people into“moral crumple zones”, whereby humans unfairly bear liability for automated decisions, as no one can be found to blame. 42 The question of liability is particularly difficult in unconscious and unintended algorithmic collusion, as explained earlier. 43 These regulations and laws on data ownership need to be developed coherently, with common and commonly-agreed principles in mind. iii. Regulatory technology or RegTech: As more data and algorithms are used in products, more data and algorithms need to be used in regulation to keep up with the sophistication of industry. The existing means of regulation are considered increasingly inadequate when regulated entities continue to Regulating AI in the Finance Sector in India

What does the regulatory responses need to be? 9 The existing means of digitise and automate. 44 regulation are considered RegTech can range from increasingly inadequate simple digitisation of when regulated entities compliance, to statistical continue to digitise and modelling for detecting fraud automate. in financial transactions, to complete automation of regulation once instituted. The RBI will soon set up a Data Sciences Lab, which will be a welcome first step towards the use of RegTech. 45 Indian financial regulators must improve their ability to monitor and regulate a fast digitalising sector. Similar recommendations are made in the report of the Steering Committee on Fintech Related Issues set up by the Ministry of Finance. 46 Regulating AI in the Finance Sector in India

10 Conclusion 6. Conclusion The promise of fintech has been one of financial inclusion. The sector has seen investments from impact investors because of this promise of a favourable impact on unbanked people. 47 This promise must not be bought wholesale. With the use of big data and AI, the very process of financial inclusion can exacerbate exclusion of the most vulnerable and can entrench monopoly and exploitation. Financial inclusion must not be seen as a panacea to structural issues of employment, remuneration and livelihoods. At the same time, caution does not mean that the impressive gains possible from AI in finance The gains of fintech should not be realised. These should be realised in gains can be realised in non- non-extractive ways, extractive ways, and the and the most effective most effective instrument to instrument to do this do this is through common is through common data ownership, along with data ownership, along modern ways of regulating with modern ways of finance. It is important to regulating finance. tackle these issues now, before widespread use of AI in Indian finance creates a path-dependence of exclusion. Regulating AI in the Finance Sector in India

Endnotes 11 Endnotes 1 Reserve Bank of India,“Report of the Working Group on Fintech and Digital Banking”, November, 2017. Available at: https://rbidocs.rbi.org.in/rdocs/PublicationReport/Pdfs/WGFR68AA1890D7334D8F8F72CC2399A27F4A.PDF. 2 KPMG and NASSCOM 10,000 Startups,“Fintech in India: a global growth story”, June 2016. Available at: https:// assets.kpmg/content/dam/kpmg/pdf/2016/06/FinTech-new.pdf. 3 YES Bank,“India Fintech Opportunities Review. 2018”, Available at: http://www.yesfintech.com/data/cms/ifor-report-2018.pdf. 4 EY,“The Battle for the Indian Consumer”, October 2017. Available at: https://www.ey.com/Publication/vwLUAssets/ ey-the-battle-for-the-indian-consumer/$File/ey-the-battle-for-the-indian-consumer.pdf. 5 In the first 11 months of 2018, 132 funding deals were signed in the fintech sector in India. Almost half of the total funding was raised by lending startups; Capgemini,“World Fintech Report, 2018”, Available at: https://www.capgemini. com/wp-content/uploads/2018/02/world-fintech-report-wftr-2018.pdf. 6 KPMG and NASSCOM 10,000 Startups,“Fintech in India: a global growth story,” June 2016. Available at: https://assets. kpmg/content/dam/kpmg/pdf/2016/06/FinTech-new.pdf 7 Reserve Bank of India,“Report of the Working Group on Fintech and Digital Banking”, November, 2017. Available at: https://rbidocs.rbi.org.in/rdocs/PublicationReport/Pdfs/WGFR68AA1890D7334D8F8F72CC2399A27F4A.PDF. 8 EY,“The Battle for the Indian Consumer”, October 2017. Available at: https://www.ey.com/Publication/vwLUAssets/ ey-the-battle-for-the-indian-consumer/$File/ey-the-battle-for-the-indian-consumer.pdf pp 32. 9 EY, ibid. 10 Mathur, Nandita,“AI startups that provide security, lending solutions”, Livemint, October 8, 2018. Available at: https:// www.livemint.com/Technology/KKbxZ3W9MeJV2H1dHSsHaJ/AI-startups-that-provide-security-lending-solutions.html. 11 Hetavkar, Nikhat,“Banks fund agricultural technology companies for rural lending”, Business Standard, June 9, 2018. Available at: https://www.business-standard.com/article/finance/banks-fund-agricultural-technology-companies-for-rural-lending-118060900031_1.html. 12 Vishwanathan, Vivina,“Banks are using chat bots and robotics to enhance customer experience”, Livemint, April 6, 2017. Available at: https://www.livemint.com/Money/B58W1vTGT9puoigm1BBtqJ/Banks-are-using-chat-bots-and-robotics-to-enhance-customer-e.html. 13 Ghoshal, Anirban,“AI app for rural India can tell if a person is being pressured to take a loan”, TechCircle, March 21, 2018. Available at: https://techcircle.vccircle.com/2018/03/21/ai-app-for-rural-india-can-tell-if-a-person-is-being-pressured-to-take-a-loan. Regulating AI in the Finance Sector in India

12 Endnotes 14 Goudarzi, Saman, et. al.,“AI in Banking and Finance”, The Centre for Internet and Society, May 2018. Working Draft, available at: https://cis-india.org/internet-governance/files/ai-in-banking-and-finance. 15 Raval, Abhishek,“AI takes cybersecurity to a new level for HDFC Bank”, Indian Express, July 25, 2018. Available at: https://www.expresscomputer.in/magazine/ai-takes-cyber-security-to-a-new-level-for-hdfc-bank/23580/. 16 FE Bureau,“Artificial Intelligence and Cybersecurity: Closing the IT security gap”, Financial Express, October 11, 2018. Available at: https://www.financialexpress.com/industry/technology/artificial-intelligence-cybersecurity-closing-the-it-security-gap/1344737/. 17 Bhushan, Kul,“Artificial Intelligence in Indian Banking: Challenges and Opportunities”, Livemint, July 9, 2018. Available at: https://www.livemint.com/AI/v0Nd6Xkv0nINDG4wQ2JOvK/Artificial-Intelligence-in-Indian-banking-Challenges-and-op.html; Bhushan, Kul,“Artificial Intelligence in Indian Banking: Challenges and Opportunities”, Livemint, July 9, 2018. Available at: https://www.livemint.com/AI/v0Nd6Xkv0nINDG4wQ2JOvK/Artificial-Intelligence-in-Indian-banking-Challenges-and-op.html. 18 Chartis and IBM,“Demystifying Artificial Intelligence in Risk and Compliance”, April 2018. Available at: https://www. ibm.com/downloads/cas/DLJ28XP7. 19 O’Dwyer, Rachel,“Algorithms are making the same mistakes assessing credit scores that humans did a century ago”, Quartz, May 14, 2018. Available at: https://qz.com/1276781/algorithms-are-making-the-same-mistakes-assessing-credit-scores-that-humans-did-a-century-ago/. 20 Pani, Priyanka,“Fintech players rely on AI to build credit scores for disbursal of loans”, The Hindu Business Line, May 10, 2018. Available at: https://www.thehindubusinessline.com/money-and-banking/fintech-players-rely-on-ai-to-buildcredit-scores-for-disbursal-of-loans/article23739768.ece 21 EY, ibid. 22 Financial Stability Board,“Financial stability implications from fintech”, June 27, 2017. Available at: http://www.fsb. org/wp-content/uploads/R270617.pdf 23 Reserve Bank of India, ibid. 24 Metinko, Chris,“How will robo-advisors do in a recession?”, The Street, March 8, 2017. Available at: https://www. thestreet.com/story/14029091/1/how-will-robo-advisors-do-in-a-recession.html 25 OECD,“Algorithms and collusion: competition policy in the digital age”, 2017. Available at: http://www.oecd.org/daf/ competition/Algorithms-and-colllusion-competition-policy-in-the-digital-age.pdf. 26 Ezrachi, Ariel and Maurice E Stucke,“Algorithmic Collusion: Problems and Counter-Measures”, 127th Meeting of the OECD Competition Committee, 21-23 June 2017. Available at: https://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=DAF/COMP/WD%282017%2925&docLanguage=En. Regulating AI in the Finance Sector in India

Endnotes 13 28 Zetzsche, Dirk et. al.,“From Fintech to Techfin: the regulatory challenges of data-driven finance”, EBI Working Paper Series 2017, no. 6, European Banking Institute, April 25, 2017. Available at: how-ant-financial-grew-larger-than-goldman-sachs.html. 29 Sen, Sunny,“India’s largest private sector lender HDFC Bank wants to use artificial intelligence, here’s how”, FactorDaily, December 14, 2017. Available at: https://factordaily.com/hdfc-bank-artificial-intelligence-chatbot/. 30 Capgemini,“World Fintech Report, 2018”. Available at: https://www.capgemini.com/wp-content/uploads/2018/02/ world-fintech-report-wftr-2018.pdf. 31 Zetzsche, Dirk et. al., ibid.; Ghoshal, Anirban, ibid.; EY, ibid. 32 EASE, in fact, was developed by Accenture and Grameen Foundation to target rural customers of Microfinance Institutions(MFIs). Several digitised MFI products exist that are aimed specifically at rural women. 33 Gulati, Gopika and Gaurav Chakraverty,“Digital Winds of Change in Rural India”, FinDev Gateway, CGAP, July 2017. Available at: http://www.findevgateway.org/blog/2017/jul/digital-winds-change-rural-india. 34 Selbst, Andrew D,“Fairness and abstraction in sociotechnical systems”, ACM Conference on Fairness, Accountability and Transparency, December 2018. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3265913. 35 Reserve Bank of India,“Peer to Peer Lending Platform(Reserve Bank) Directions, 2017”, Master Directions. Available at: https://www.rbi.org.in/Scripts/NotificationUser.aspx?Id=11137; Reserve Bank of India,“Guidelines for Licensing of Payments Banks”, 2014. Available at: https://rbi.org.in/scripts/bs_viewcontent.aspx?Id=2900; IRDAI,“InsurTech – Working Group Findings and Recommendations”, July 2018. Available at: https://www.irdai.gov.in/ADMINCMS/cms/ frmGeneral_Layout.aspx?page=PageNo3519&flag=1; Department of Economic Affairs, Ministry of Finance,“Report of the Committee to propose specific actions to be taken in relation to virtual currencies”, February 2019. Available at: https://dea.gov.in/sites/default/files/Approved%20and%20Signed%20Report%20and%20Bill%20of%20 IMC%20on%20VCs%2028%20Feb%202019.pdf?fbclid=IwAR3T5zNI8QGkaaJ4rRUdmcV6AHwUtFZIds5MhRaLv9QiEt6Rtjkq2CvuC2M 36 ET Bureau,“RBI to form regulatory sandbox for fintech, set up data science lab”, September 21, 2018. Available at: https://economictimes.indiatimes.com/industry/banking/finance/banking/rbi-to-form-regulatory-sandbox-for-fintechset-up-data-science-lab/articleshow/65894188.cms 37 Saman et. al., ibid., and at https://cis-india.org/internet-governance/files/ai-in-banking-and-finance 38 Pahwa, Nikhil,“Why the RBI wants a Public Credit Registry with Aadhaar and CIN linkage”, Medianama, July 7, 2017. Available at: https://www.medianama.com/2017/07/223-public-credit-registry-india-rbi/. 39 White, Gillian B,“Can the flaws in credit scoring be fixed?”, The Atlantic, January 10, 2017. Available at: https://www. theatlantic.com/business/archive/2017/01/credit-score/512702/. 40 Ananny, Mike and Kate Crawford,“Seeing without knowing: limitations of the transparency ideal and its application to algorithmic accountability”, New Media and Society, vol 20(3). 2018. Available at: https://journals.sagepub.com/doi/ abs/10.1177/1461444816676645. Regulating AI in the Finance Sector in India

14 Endnotes 41 IRDAI, ibid. 42 Elish, M. C.,“Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction”, We Robot 2016 Working Paper, March 20, 2016. Available at: https://ssrn.com/abstract=2757236. 43 Ezrachi, Ariel and Maurice E Stucke, ibid. 44 Arner, Douglas, et. al.,“Fintech, Regtech and the reconceptualising of financial regulation”, Northwestern Journal of International Law and Business, vol 37, no 3, 2017. Available at: https://heinonline.org/HOL/LandingPage?handle=hein. journals/nwjilb37&div=17&id=&page. 45 ET Bureau,“RBI to form regulatory sandbox for fintech, set up data sciences lab”, Economic Times, September 21, 2018. Available at: https://economictimes.indiatimes.com/industry/banking/finance/banking/rbi-to-form-regulatorysandbox-for-fintech-set-up-data-science-lab/articleshow/65894188.cms. 46 Department of Economic Affairs, Ministry of Finance,“Report of the Steering Committee on Fintech Related Issues”, 2019. Available at: https://dea.gov.in/sites/default/files/Report%20of%20the%20Steering%20Committee%20on%20 Fintech.pdf. 47 Bhalla, Tarush,“India’s most active investors poured funding into this sector in 2018”, Yourstory, December 5, 2018. Available at: https://yourstory.com/2018/12/india-startup-funding-active-fintech-investors-2018/. Regulating AI in the Finance Sector in India

About the authors Jai Vipra is an independent researcher who previously worked at IT for Change. She researches the digital economy, focusing on regulatory debates in the following areas: fintech, AI, platforms and data. She is also part of advocacy efforts in national and international fora on digital trade and data rights. The views expressed in this publication are not necessarily those of the Friedrich-Ebert-Stiftung. Imprint © 2020 Friedrich-Ebert-Stiftung India Office K-70-B, Hauz Khas Enclave| New Delhi-110016 India Licensed under a Creative Commons License Attribution-ShareAlike 4.0 International(CC BY-SA 4.0) Responsible: Patrick Ruether| Resident Representative Mandvi Kulshreshtha| Program Adviser T:+ 91 11 26561361-64 www.fes-india.org FriedrichEbertStiftungIndia To order publication: info@fes-india.org Commercial use of all media published by the Friedrich-Ebert-Stiftung(FES) is not permitted without the written consent of the FES. The Wicked Problem of AI governance_ITfC https://www.fes-asia.org/ Friedrich-Ebert-Stiftung(FES) is the oldest political foundation in Germany. The foundation is named after Friedrich Ebert, the first democratically elected president of Germany. FES is committed to the advancement of both socio-political and economic development in the spirit of social democracy, through civic education, research, and international cooperation. This publication is part of a series on Artificial Intelligence in India, a collection of papers developed under the Socio-Economic Transformation program coordinated by the FES India office.