hold on the european union to open direct talk with russia. president vladimir, if the suggestion is taken up, it would mark a change in strategy for brussels ties with moscow. are stream coming out in the information you share on the internet help computers predict how you will both. that's in our documentary series dock film, looking at the role of artificial intelligence in election. they choose yes. can you hear me now? yes, we can hear you in germany. we bring you back or you've never had before. just so what, what is, who is the medical really want me and want to people who is full along the way. myers and critic to join us for macros last the

aah! for almost a century posters have been interpreting public opinion ah, when you like coming there method, a straightforward ask a small group of people questions and then use that sample group representation of the opinions of the entire population. it's a nice meeting where you're welcome. good night. thank you for coming by but in recent decades, the public has soured on answering questions, calls for doorbells. and instead of being available at a single point of contact, people have become increasingly mobile, which makes traditional pulling more challenging me today. people are expressing their opinions and habits through mobile and internet technologies like smartphones and social media. that avalanche of information is creating possibilities for new methods to explore with the public thinking.

ah, ah, me today data is available everywhere. credit card information, loyalty card information, demographic information, and community organization information. think about the things you sign up for. think about the events you go to. and when you click that little box that says, i have read and agreed to the terms of service that is contractual. several screens of stuff that you didn't read, nobody ever reads, it wouldn't really matter if you read it anyway. so that means we've got information from all over the place, and that's actually what makes it quite valuable. many people give google their email data by a g mail, their video data via youtube,

their physical location, by google maps. so when you connect all that data together, it's remarkable. the intimate picture it paints to the individual, which i actually think eclipse is our own memory. and that we forget all the places we go to the the challenge is knowing how to extract meaningful information from what looks like random noise that were applied, artificial intelligence, known as a, presents both promise and risk. while dystopian science fiction dreams up worlds where machines rule over humans, a r is more commonly understood by its engineers as the ability of a machine to learn and process information. the, they consider it a human guided tool that learns by digesting massive volumes of data, like social media, for example, and then identifying patterns and the noise. what if they were in a,

i could use those patterns to predict how humans would behave in the future. like say, predicting the outcome of an election. ready campaign for the next canadian federal election has just kicked off. ready while prime minister jackson trudeau is liberal party, cope's with a justice scandal and the ouster of 2 ministers, the conservatives, new democrats and black cubic law, all have new and untested leaders. despite as tight 6 week campaign, the parties are slow to outline their platforms. it's unclear voters will make their choices based on the issues the party or the leader in canada, capital ottawa, a small startup, is using ai to predict the upcoming election. people are calling to find so much and yeah, i'll tell you. i feel like people with people that means this is like the 1st one where they're actually looking at the policy, not the leaders to,

to front runners. they mean nothing to me. yeah. it's really about who i think is going to invest job for the economy for the environment. and i had heard a lot of people, obviously economy is always the top for everybody. but who has the best policies in physicist kenton, white and accountant, erin kelly, co founded advanced symbolic or a s i. in 2015. the market research company uses public data and a i to forecast consumer attitudes. you know, in the era of cambridge analytics and all that, people think it's some sort of science like we have a shame and making things happen. i want people to understand that nobody is, this is just math and science, but anything else and it's problems probably. it's just, it's just like, it's always been except as being done by poly and that is by phone operators. how they can use a similar approach to predict political opinions before be kind of hard to get

information about millions and millions of people and apply statistical physics to it. but now i've got facebook. now i've got twitter. how do i use that though they don't work for either political parties for news outlets. kelly and white saw the electron prediction game as a way to promote their research method. think about when we do traditional market research, how many times we interfere with the subjects with people as we're studying them, we interfere by asking them to participate. we interfere in the results by asking a question and how we phrased that question. so our whole idea, we saw that with social media was, came a great opportunity to observe people in their natural environment. and aren't you going to get much better policy? and a much more utopian society, if you know exactly what the people in that society want. and i was inspired by

science fiction growing up like as a most foundation series where you can use physics since physical physics on people and populations and map out the whole course of the galaxy. i was quite disappointed when i learned that that's not really possible. the explosion of social media offered white a potential treasure trove of data. 2 he could apply statistical methods from physics to crunch what might seem to others like messy and unusable sources of data. it was the birth of the a r s i cause cali. tale. it was sure for politics. so that's we just happened to call or we're going to car polly. we didn't put any thought into it except politics. you know, poly is made up of number of different part little programs that are crawling the web and collecting information. these programs contain complex

algorithms which help them process the huge amounts of data. you've got an algorithm to say, when do you pick someone? when you don't little algorithms analyze that person come up with a person's male or female, richer poor, young or older, black or white algorithms that allow us to find people that are talking about the topic we want to measure. if i showed you any one of these algorithms and how it work, cases are, you'd say, that's pretty simple. it's when you take dozens and dozens of put all these things together, that something more complex emerges. me, she's using more data than me or you are pretty much anyone here could understand analyze what is it that makes poly intelligent?

and notice that if they sent you, they said intelligence. so she is intelligent because she can grasp the concepts, mathematical concepts, and see patterns in people that we can see ourselves the but that description, the critics of a are uneasy method comes across as a black box outside which leaves not experts at a disadvantage to challenge the inner workings, or if the conclusions in the face of skeptics as i saw the brakes referendum in june 2016 as a way to test their algorithms. citizens of the united kingdom were voting to remain in or leave the european union. traditional polls conducted almost daily revealed, the country was split down the middle of the voting day approached. it became clear that undecided voters would likely determine the

country's fate. meanwhile, polly was sifting through new k twitter data to predict user's opinions without contacting them directly. polly was she remained right up until 3 days before the actual referendum had 50 percent voting for remain 40 percent voting for just like everyone else in the world. then there was a fascination of jo cox labor party, n p. jo cox, who had been campaigning for remain, was shot and killed by a foreign extremist on june 16th 2016. 3 days before the referendum. we wake up. we check in with polly and suddenly britain is going to exit the that it was different from what she's been saying. every day this week. throughout this campaign, i was worried because an assassination is

a black swan event lacks one. that is a rare event. one that my a i have never seen, none of the elections that i had trained her on had there been an fascination assassinations are rare. so we told her ignore the assassination of jo cox. what is the result remain? test again, jo, concert assassinated exit. what had changed when she seeing that everyone else in the world was missing? the british pound claims to a 30 year low and world stock market convulsed on the news that the majority of u. k. voters, one out of the european union, the 2nd largest economy in the world. the tale absurd things that we wouldn't even think to observe. we didn't ask her to observe the space. well, we started going through david cameron had dental many press conferences during that time. and there was one line is something he said,

it's right that we're suspend campaigning activity in this referendum. and every one source will be with joe's family, with her constituents. at this terrible time, she noticed that throughout the year that she was observing people. if people came on line and they said, i'm thinking of voting for an exit. if they had a network of people around them, friends and family, who calmly talks to them about why we should have a remain and what the consequences would be over time. that network was effective in winning these people over to remain if they kept the dialogue going. but once people weren't talking anymore, a certain percentage of those undecided people would then go exit based on what she had noticed in the past. so she applied that algorithm. she got it bang on ah, to better understand polly's methods. it's helpful to look at what she has in common with traditional polling.

ah, one way of measuring public opinion is by interviews. brief interviews are called hold of you. people are carefully selected as samples of the important groups of the general public. for decades, posters successfully reached people by landlines. direct mail, we're knocking on doors. get 80 percent response rate, so all our calls were landlines. we didn't really have any cell phones. and people love to participate in service. you know, getting it wrong, getting an election wrong would be like, you know, falling out of a boat and missing the water. that's the 3rd now knock and the house looks dark. yeah. they're not home. but in the last 30 years, landline use and poll response rates have plummeted. you phone or a landline in any house, you're much more likely to get an older woman than you're to get anybody else. millennials, if you want to get millennials, you're probably not going to get them using landlines. you're probably going to get

them using cell phone because as likely or you're likely to get them online. traditional posters need to diversify their methods to reach different audiences. what you're trying to do is represent the political marketplace, the important political marketplace, the voting population, as accurately as you can. so what you need to have is good coverage of all of these different groups of the population. and then what you do is you try to look at them in aggregate to cover the canadian population in 2019 poster daryl bricker, combined landline pulse with cell phone pulse and online surveys. but polly's a takes a different tech. as our population gets more diverse, we actually need to have a bigger sample. i mean, candidate is very different now than it was in 1970 or 1959 when we 1st started doing phone pulling. but we haven't really evolve the science since the 1950s. so we're still getting the same sample sizes. we're still approaching the science in the same way, even though the population is changed a lot. now with social media,

finally, we have the opportunity to involve this science for the 1st time in 50 years. ah, traditional polling has struck out on several recent election predictions. we're going to win the white house. we're going to take it back. perhaps most famously in the 2016 us presidential election posters overwhelmingly predicted hillary clinton to be donald trump. if the lines are long tomorrow, please wait me trumps. when demonstrated that posters had missed the boat, they predicted the popular vote but missed the tight local races in swing states, which delivered trump an electoral college victory. clearly their models were just completely off. the actual percentage of the vote predicted by the polls is actually what happened. but we weren't measuring the left for college.

obviously, you're going through, you know, a dark night of the soul. whenever these kinds of things happen. it's too expensive and inefficient to pull regularly, by traditional methods and each electoral district known in canada as a writing. but social media is huge, scale makes that possible with a i talking about politics on social media or all your opinions on social media is now kind of the norm like that's become a normal way to communicate in the same way that people used. i think long conversations on the phone frequently to now, of course, back in the day you can just tap people songs. you had to call up and ask them, but opinions but, but now the thing with social media, this interesting is when people are communicating their opinions on social media, often they're doing it in a form that is public. and that allows us to see what they're saying and to then derive their, their opinions from that. but not all social media platforms have data that are relevant to white. we use a sample for data source, depending on the problem that we want to look at. we have found that for elections,

we get the strongest signal from many posters and social scientists question the value of data from twitter platforms like twitter and facebook is basically a group of self selected people. because not every canadians are on social media. and even they're on social media, got all of them, use it to discuss politics. so get it. and basically you're getting a slice of smaller slice of the public. i think it's like 12 or 15 percent. use twitter for news 20 percent. use twitter overall in my own lay understanding of that. i tend to think about to lease journalists and like high engaged, super participants that are on twitter. so not everyone is on twitter. let's take the loan number, 17 percent. that is still a whole lot bigger than the current 9 percent or less. you get on diets. 17 percent of a country of 36000000 is about 7000000 people that we have

access to white things that a i can fit through the large. but typically young and urban user base of twitter and canada and nonetheless, make forecasts that are representative of the entire population. less than 3 weeks to go until election day. and i know at some point your artificial intelligence pulling system comes to a conclusion about what you think is going to happen on the 21st of october to early still, i presume it's too early. what both? well for mr. crudo is that the liberals have consistently started this campaign. been in 1st place, 1st place in the popular vote, or total vote, i guess, as we call later, or c count speak count. so that's the difference. okay, good. we're doing each individual writing so, and that's as we saw on the on terry election. it makes a big difference just because you're leading in the total vocab in the polls doesn't mean you're going to win that vote efficiency. yes is what was where is

that in a 1st post past the whole system. that's right. because most people that work with data from lots of people and they will say, i want to collect everything everything. everyone is saying on facebook and twitter and read it. and look at it all. most critics assume i'm withdrawn what is called the twitter firehouse. if so, why we need a way to filter the huge volume of real time data and collect only the data useful to predict elections. given the magnitude of the challenge predicts dispute my optimistic claims. now if i all i did, capitan that fire pull in that stream of data, you're going to get a very skewed distorted view. that's why we don't count, treat people are big idea. is what the pollsters have been

doing. works actually secret you make really, really good samples of people. in other words, rather than trying to clean up the twitter, firehose white claims that he has found a way to bypass it before the holiday collects any actual data. poly 1st builds a representative sample, just like posters, those who say that you can look at social media and get to representation and what's happening at the writing level. it's an interesting concept. we're certainly exploring that ourselves. what i bet the farm to morrow and being able to do an election in which i predict every single right incorrectly based on social media. know me. first of all, tracing people back to the writings difficult. it's difficult to figure out where's that account tweeting from, you know that old joke the, on the internet. nobody knows your dog well on twitter. nobody knows whether you

are in china, russia, or right here in toronto. it's very different to say yes for sure. that accounts come canada, a representative sample of canadians must consist of actual canadians. white thanks, he can confirm his twitter. users are legitimate. we use the information that people put on their account. so we see that they're in canada. great. they say i'm in canada. so verify that 1st test is your friends also in canada. look at your history. look at what you're talking about. you're canadian. probably talking a little bit about maybe hockey or other canadian things. if you're talking about the cold weather me for posters. a sample is representative and random only when every member has an equal chance of appearing in the sample. imagine that you had a whole big bowl of jelly beans and there was a red, one, blue ones,

and green ones, and yellow ones. you don't mean to count every single jelly beam to figure out that there's red ones when she was whatever the color is, where i said, if you, if all of them are, are equally distributed across the entire big ball of jelly beans. if you stick in your hand and you pull it out and you put down those jelly beans, you should have some representation of what's inside the jar. if you do it 10 times, you'll have a pretty good representation of what's in that you're not because the jars getting smaller, but because the samples become consistent. so that's what sampling is really all about. the historically, poster sampled responded by randomly selecting names from a phone book in the digital world random and representative sampling requires a different paradigm. is a game called 6 degrees of kevin bacon and says,

i'll pick a movie actor and try to link them back to kevin bacon in 6. and you can play the same game on facebook and twitter can pick any person any celebrity, and tracks. link yourself back to them in an average about 23, and 4 connections. anyone in the world always algorithm uses a mathematical computation to guarantee an average minimum of hawks between the twitter users that collects. that's how it defines virtual distance and randomness between users. the critics argue that still doesn't address the lack of diversity among twitter users. i can account for that bias. twitter. i do it by saying this is what the census is says, how my distribution should be in the algorithm. you say it differs too much from what you expect and you have to wait until balances to for example, when i see that my sample starting a little bit lopsided, young women now just stop collecting younger. now,

you not add any more into your sample to you start to get older rural female. so instead of looking like the majority of twitter, poly sample is limited to a mix of users that match the actual demographics of each writing. why claims this addresses? what critic fee is a tragic flaw and polly's main source of data. and that is the key algorithm behind poly. if ever a bombshell hit a canadian election campaign, it happened this week. photos of justin crudo in black and brown face swirled a very complicated broo of opinion in the country. aaron kelly, what is poly telling you about the fall from all of this? it was a big show. so polly saw mr. sure those numbers the number of seats seek count to fall by 25 seats yesterday though he is rebounding warning. he's up 5 in that of 20 seat losses, but it knocked him from majority territory into minority territory. jake,

me saying the n d p leader was the biggest winner yesterday. we saw his, his poll numbers go up. so he has really, he really shot through this. so how does an algorithm go from tweet to projecting a drop in seats? what is a tweet actually mean? what does it mean when we're tweeting? we know people are making jokes in error there i, ron, people are showing more images, and that creates challenge. it's harder to interpret. means which of these shorthand graphics are often pop culture references. it's not like, i don't know what's going on here and then i'm like, well, i don't understand this and my computational power is a living, breathing human. what is his poor computer? you know, he's going to have to make sense of this. according to white, poly is not trying to interpret the meaning of tweets at all. so my 1st few for a's into elections forecasting. i ran the experiments and consistently,

what i found was just the plain old mentions, outperformed the positive or negative sentiment every time. ready all the tracks, the percentage of users in the sample who mentioned each federal party. then the a i compares those results to historical opinion patterns. after a scandal broke, involving an old photo of the prime minister in blackface mentions of the liberal party spiked in relation to the others. without understanding the meaning of these tweets. polly interprets these changes and mentions as a drop in projected seats for the liberals. so i think paul's are interesting because they're a flash point in the past, right? they don't predict the future there a moment in time in which you're getting a flavor of what the public is sort of thinking about the a i collect data constantly that makes it possible to identify patterns and user activity over time compared to poles that are held less frequently, one of the biggest, most difficult things and point when you're doing these tracking. you see something

happen for good or bad. they say, well as one piece of data is going to do, then you have to wait another day. is that real or was it just a blip here? always sort of in that in that feeling of how many points makes a trend, is it one? probably not. is it 2? maybe 3. yeah. for for sure. but you know, where is that point? in order to identify a trend poly use as public polls as a reference point? she analyzes patterns of user engagement. whenever her selected twitter users mentioned political parties. then she compares engagement with each party standing in the pulse. based on those relationships poly predicts, if her data indicates a trend is stable or not people, sometimes they'll be angry about something about an issue and they might be angry about it in a moment of time, the poly is able, in a very unbiased way to see through that anger and say, how long will it take the population to get over this anchor and this hurt and will

it carry long enough to election day? polly tracks twitter engagement constantly. that continuous stream of activity is referred to as longitudinal data. by studying data in this way, poly can identify patterns over time and then compare them to present activity. it also means poly can look backwards in time at the data to re examine unexplored trends. for the brown case, black scandal. i don't think there was a single poster that asked a question if you saw a party leader in a questionable costume in their twenty's. pretty change your opinion. i don't think anyone ought to ask that question to the wonderful thing about poly and the launching studies is that i can go back. i can get an accurate measurement of exactly how you thought a week ago or a month ago. and that's something you can't do

a week later or 2 weeks later by asking a question, i would suggest that perhaps, canadians not really care about blackberries as much as they perhaps profess due on social media. there was a lot of outrage and i would even call it performative at rage on social media in particular. but being like, oh my god, i can't believe this. i was like, really, you can't believe those. you can't believe that, like i'm, or, you know, a rich white guy from came back would have dawn to black face at a receipt private school that had a party themed and ruby and nights like, i mean the party in and of itself without the black face was racist in or given how massive that story was when it broke a week and a half ago. it has virtually, i want to say disappeared from the media, but you see so few references to it. now, is this story essentially over to the best that you can tell through your polling?

yeah, i think to keep bringing it up for the other parties. keep bringing it up. would be beating a dead horse at this point. i think he is recovering. and i think the slower recovery might be due to you know, a mix of issues and not just blackface juggling, saying really impressed a lot of people with his initial statements after the black face scandal broke, that goodwill is done now. and as in gone, well he's up to seats from before blackface, but that's down from having been up 9 before. so it's starting to, peter owed me like artificial intelligence and machine learning or so i use in different ways to mean different sorta magical things about computers that think the way humans do model. meaning pretty much is just the very sophisticated and very flexible algorithm which allow us to process very large amounts of data with, with a very complicated relationships between them. and have very flexible models which will allow the computer to sort of figure out,

unlike the races. why or why not just statistics or collection of coke? the difference is that she's able to do this mostly on her own. i don't have to go in and write a rule and say now poly, when you see this, do this, she's able to figure things out on her own. polly's ability to learn is the basis for how she predicts future voting outcomes. me, the key to machine learning is a process known as training a statistical model. so we talk about training a model. we mean, we have some data where we already know the answers that maybe it's how people in the last election, or maybe you know, how many people did or didn't buy guns, but we already know the answer. and then we use that to drive friends as a, okay, it seems like men who live here are more likely to vote for that person or many subscribe to this magazine are more likely to buy a gun and we learn those relationships. and we say,

let's see if we can use our model that was trained on a previous data to try to predict what's gonna happen next. ready election night 2019. ready almost $18000000.00 canadians have cast their ballots, including the advanced pulse that represents a turn out of 66 percent, several percent higher than expected. ah, the liberal form, the government. but the conservatives win the popular vote. days later, it's time to compare the actual results with the seat predictions federal election, 2019 wrapped up of course this week with a liberal minority governments. this is the national seat projection that you gave us would show the liberals in arranged between 116 and 173. but essentially at 145, the tories at 123. the n d p at 48 the block at 46, and the green somewhere between one and 5, and you have the possibility there of

a couple of independence and a people's party. now let's flip that over. and this is what the national seat projection was on sunday. so the last one was saturday. this one is sunday, the day before the election. and you can see that things have changed. you've got the liberals now at $155.00 tories that $118.00. and then let's roll monday. and this is what the actual election results were. 157 liberal seats, 121 conservatives. 32 block, 24 new democrats. 3 greens, one joey, wilson, re bold and nothing for the people's party. if you ever want to evaluate how well someone did predicting election, i think the fair thing to do is to compare them to other people who try to predict the election company better than all the other companies. and that's probably a pretty good result. the day before the election, some traditional posters were only predicting the popular vote. others were also willing to predict a liberal minority, but few publicly predicted

a narrow range of feats ah, according to pollster $338.00 canada. that correctly predicted the outcome of 299 of the 338 writings. according to advanced symbolic poly correctly predicted 308. was this quote unquote an easy election to predict. it wasn't an easy elections predict, but it was predictable, right? and i find it, you know, it's interesting because everybody was saying was so close, that's the, that's why being able to look at the seat count and do the writings is it's becoming almost a necessity if you want to do pulling. we saw it at 1st with the trump election in the states, and that's been consistent. it seems that elections are getting closer and closer. and so obviously when it's a minority government, it's little bit harder to call. but this was consistent for poly that she always thought as a liberal when you look at our election results, we did really well in the 2019 election. but there were some writings we got off.

it comes down to we're making assumptions that all parties are campaigning equally on the last weekend before the election in all writings where they might have a chance. all the political scientists out there are probably laughing at me right now. it turns out that's not true. turns out political parties take writings that strategically they want to win and we haven't taken any of that into account. it's why i would love to have a really open dialogue with the people that know the political science. both white and kelly probably claimed that polly's final c prediction was more accurate than most established posters. although what does it mean, a lack of expertise and how campaigns work to critics of an a i method this

admission reveal some naive and damaging political assumptions. but to supporters, holly's relative accuracy and projecting seats shows great promise for a whenever i hear people say, i've got to sit back and say ok. * not the thing well, you know, the big publicly traded research corporations that have more money to do this kind of stuff than anybody else has to do it. not to mention that google's and the in the facebook's, in the twitters and all these other people who have all of this money and all the best i've noticed minus all working on this, trying to think they don't. they can't predict based on their platform better. and pollsters, and that is tissue still demand more proof. they continue to doubt the value of twitter as a primary data source. and they insist that poly established a much longer track record of predicting elections before they concede this method is reputable and accurate. so the common saying this uses a defamation,

people by products and on this as past performance is not an indicator, a future performance. and i think that applies to predictive algorithms as well if they've done well in the last couple actions. i think that works in their favor, but still not a guarantee and as they do better and better and more and more elections, i think you can start to trust them more and more according to a s i. polly's prediction model will only get better with more election data and with input from social scientists to refine their algorithms. we've come a long way to get to where you're at today and a lot of what has helped a i get to where it is that today is the internet then explosion of data. but the explosion of public data comes with risks. social media is free exchange for users, privacy. personal data is then available for marketing. critics also warning that platform, they're using the data against their own users by making the platforms more

addictive. when the car came out there was on the belt, there was no airbags, but when we discovered that, oh my gosh, these carson going to acid and they can run people over. we are demanding better breaks. we start demanding airbags, we start demanding seatbelt. the 1st 1015 years of social media, the argument was let's put handcuffs on this company because we want to see where and what they can do and how to help society. but we're now discovering all the car accidents. what is covering all the harms that they can do? so for need and i think the next few years is going to be debating what does regulation especially look like. i think the big problem is we're thinking of campaigning in an analog world. when in fact, it's all automated and digital, and neither our language nor our laws really anticipate the power that facebook has, in the way in which individuals are political parties or foreign governments can use that power to tip the scales of the election. if not just muddy people's

perception of the election, because the whole point we have an election to legitimize the people believe rightly or wrongly that algorithms have taken over the election. well, then they're not going to see the results of legitimate. they're not going to see the government as legitimate, therefore it's going to make government's ability to actually govern that much more difficult in many ways, using data to get people to vote and pulling people to participate in politics is fairly beneficial. in fact, if you had asked me what could data be good for, it's like getting more engagement in politics. and so i don't think that by definition, political advertising is a problem. but i think is that the way that online advertising works in the fact that requires so much surveillance to develop ad profiles and that the way you're doing targeting so much care fraud. those are the kind of real issues. i think canadians are feeling creeped out about the way in which technology and

surveillance is playing a growing role in their society. but i think they feel powerless. so we've had this notion of, if you want privacy, stay at home. if you want privacy only tell, but if you decide to participate in the public square, you put information out there you will be held to have said it's okay for anybody to do anything with that information. and that is what just does not work in the modern world, i think is really quite pernicious to think that will, will solve this with consent. and we're just going to change the consent in or consent out the fly. and that is that still the individual against this massive organization. and the great divide is they know a lot about you. you don't know a lot about them. we invented all of these structures and we can invent something else. polly's election predictions depend on the ongoing popularity of twitter and easy access to its data. but if new regulations to protect user privacy or restrict

the social media business model came into force, they could profoundly change the way people use social media. i don't believe social media is going to become more private. there will be a need for public networks where people can go create content that anyone can view and anyone can engage with kind of making a big debt on it to me. i don't know. i think maybe the traditional posters are afraid of this because it's the future. there was a famous twitter shot that microsoft put out the name of the chat box, but it was a demonstration of language. and it was a really, really sophisticated, wonderful lot of people got wind of this chat box. her calendar started saying,

what can we do to teach this a i to be the most offensive repugnant twitter account on the planet. and they succeeded. little a, i became a horribly racist phobic thought microsoft as a shot. why do we as people feel the need to be miserable? mean and that bad to the heart and show that example to the machine. ah, when you see bad children usually. so when it comes to a i, when it comes to all of this, i got to me i north star. i have to be a good parents. have to be the best parent for polly's me.

i oh i i oh, the finally he can take the trip again. ah, my normal 9 days is doing he's 99 pilgrimage on the way of saint james. during the corolla virus pandemic with close to thousands of pills, the now minority has rediscovered his why disease because the way of st. james is a journey, not a destination. focus on your out 90 minutes on the w.

ah me. oh oh, i can. because you oh no. no.

the news . this is the w news lies number lead is the european union about to open a direct line of communication with russia. president flattery recruiters, germany chancellor angle americans calls on the e. u to begin talking to the kremlin to address what he described as russia. quote, many provocations against it. also coming up in hong kong, last pro democracy, newspaper silent people line up to buy a copy of the final print edition of athens,

bailey. we will find out what this means for the cities opposition.