Artificial Neural Networks

It was 1990, the second year in Sydney. I was working for a fintech startup buzzing in a small stylish office in the Little Buckingham Street, Surry Hills. The Little Buckingham was a charming leafy street with cottages and warehouses, occupied by fashion ateliers, art galleries and alike. 

The Internet did not exist, mobile phones looked like bricks, cheap Taiwanese made IBM personal computer clones with DOS operating system were emerging.

Windows 2.0 was released two years earlier in 1987, and Lotus123 was the most amazing spreadsheet application we had ever known. 

Our company had a joint venture with the PC manufacturer Olivetti; we were using several Olivetti machines in the office. Olivetti had a modern office on the William Street. There, we carried out a demo integration of our financial hardware with a banking application. 

OCR (Optical Character Recognition) technology had been available since 1960s. I was integrating OCR cheque readers to our products. These motorised machines captured cheques and read text printed on them. However they were incapable of resolving handwriting. 

Lunch break starts. I used to walk down to Broadway through a long pedestrian underpass under the Central Station. At the time Broadway was bustling with a cosmopolitan crowd. Long passed its heydays, it had Sydney University and UTS students, bookstores, Chinese takeout shops, sex-shops, antique shops selling world war memorabilia, helmets, medals and bayonets, shops selling tents, hunting equipment, knives and guns. A bookstore called Coop sold discounted books for uni students. 

On a sunny afternoon I was strolling inside the Coop bookshop. I was looking for an interesting book. I picked up a hardcover on neural networks. 

I was curious if computers could recognise handwriting using Artificial Neural Networks (ANNs.) 

ANNs imitate biological neural networks of a human brain. I found a book that had variety of examples on the subject matter. 

Initially ANNs were constructed using Digital Signal Processing (DSP) hardware. Later on Graphical Processing Unit (GPU) hardware took over. 

This was my first glimpse at the world of AI, without knowing ANN was the foundation of AI. 

You could develop, I thought, an ANN computational model that would accurately recognise handwritten characters. It should be able to differentiate similar characters, say ‘1’ from ‘7’, handling nuances in handwriting. This way you could feed a handwritten number on a bank cheque to an ANN hardware and resolve its value. 

Suddenly it occurred to me, an ANN on a chip is no different than its biological counterpart in human brain. 

Like you teach a child how to write, you need to teach the ANN how to read handwriting. This is called “supervised learning”. 

Teaching an ANN hardware involves scanning the handwriting and feeding the information to an ANN processor. The ANN outputs what it recognises. A human agent verifies (supervises) if the ANN’s recognition is correct or not, which is then fed back to the ANN. Verification would make the ANN to re-adjust its internal “weights” on the patterns it “saw”. The cycle continues until the ANN gets sufficiently better in recognising all characters, in all varieties of handwriting. 

The more you expose subjects (child or ANN) to diverse sets of handwriting, the better they would become. 

ANNs or humans initially need supervision, but at a point they could keep learning unsupervised and differentiate not only their own (supervised) handwriting but others’. This is called self-learning. 

There may be illegible handwritings. Those may have a digit or two impossible to work out. In those circumstances an ANN would be no better than a human, it will fail. 

An ANN would be vastly accurate in its reading than a human’s, if it was trained hard with much larger and more diverse datasets. Nevertheless it may produce inaccurate results no matter how hard it was trained and how small the chances are. On a bad day even Einstein would have failed. 

As I read the neural networks book I wanted to run experiments and verify my understanding. But I couldn’t go pass thought experiments. Computers with GPUs did not exist then. 

Twenty years later, in 2010, in a different company, we kickstarted an anomaly detection project that would pinpoint fraudulent transactions recorded by our Payments product. I worked with a data scientist who developed supervised ML algorithms. I implemented those algorithms using a statistical analysis programming language called R.

The problem of inaccuracy remained. An AI model may fail in peculiar ways no matter how hard it was trained and fine tuned. Sometimes we had false positives (a normal transaction was flagged as anomaly), or we had false negatives (a fraudulent transaction was treated as normal.) With fine tuning, accuracy rate could be improved, but it could never reach 100%. 

ChatGPT is a chatbot, based on an AI-powered language model developed by OpenAI, capable of generating human-like text based on context and past conversations1. 

Agent 007 decides to battle against an eccentric scientist, Dr. No, who is determined to ruin the US space programme.

On a beautiful day in mid 1960s my older brother decided to elevate me from being a useless little brat to a more respectable brother status with no reason. We went to a stylish movie theatre in the main boulevard and watched Dr. No. 

I was mesmerised by Dr. No. I wondered how come Dr. No, the coolest villain ever, be utterly ruthless and in control at the same time. In the end Dr. No’s world domination plans were spoiled by Agent 007; Dr. No attempted to stop him, but fell into the reactor pool, boiled to death.

In Asimov’s Foundation and Empire (1952) there is a villain called the Mule - my all time favourite villain. This guy could telepathically read and manipulate anybody’s mind in the entire universe and make them scared of him. This way he could take over planets at rapid pace. Next to him, Dr. No looked like a jester. 

It may be possible for an AI bot to behave like a super-powerful mind reading bully. Welcome MuleGPT. 

When OpenAI released the large language model GPT-4, in March 2023, it was good at identifying prime numbers. When the AI was given a series of 500 prime numbers and asked whether they were primes, it correctly labeled them 97.6 percent of the time. But a few months later, in June, the same test yielded different results. GPT-4 correctly labeled 2.4 percent of the prime numbers. AI researchers prompted it with—a complete reversal in apparent accuracy 2.

AI enthusiasts speculated. With GPT-4, it is possible that the OpenAI developers were trying to make the tool less prone to offering answers that might be deemed offensive or dangerous. These changes required fine-tuning, ie. training the engine with a different set of datasets. 

Fine-tuning could have induced side effects in prime number detection, akin to random mutations causing undesirable effects in biology. 

As a consequence of mutations the gene may produce an altered protein, it may produce no protein, or it may produce the usual protein. Most mutations are not harmful, but some can be. A harmful mutation can result in a genetic disorder or even cancer.

Hence, it is possible that MuleGPT, an AI powered language model chatbot, may unexpectedly change its behaviour and become dumber, as a result of fine tuning. 

If we are lucky, MuleGPT may decide to leave the planets it conquered and jump to a blackhole for a swim. 

---

1. ChatGPT blog ↩︎

2. Scientific American - AI models can get worse over time ↩︎

Datça - the last frontier to Eden

I am at a modern coffee shop in Datça, sitting inside, behind a large, stylish ventilator. The shop’s name sounds promising, “The Coffee Grinder”. 

I am reading a Turkish book written by Özgül Tuzcu and Ezgi Kurt New Life in a Seaside Town, interviews with 60 individuals who settled in Datça.

I order a small cup of Americano. The coffee is not freshly ground and it is served boiling hot, leaving a veneer of plastic-cardboard flavour in my mouth. They serve coffee in disposable cups here.

Last time I visited Datça, it was in 2018, five years ago. 

Since then the town has grown. There are more townhouses built outside the town centre. Locals are concerned with environmental impact of new building projects. Builders leave not enough space between buildings. Diminishing air corridors and climate change are suffocating Datça.

Datça locals are worried about their town is in the footsteps of  Bodrum. Once a small fishing village, Bodrum lost its charm long time ago. It turned into an overpopulated, overdeveloped, congested and polluted mega town, a familiar tale for many coastal towns all around the world.

In the last half a century, increased global wealth, the rise of Internet and cheaper air travel ramped up tourism by 56 times (https://ourworldindata.org/tourism.) Before the pandemic nearly 4 million tourists travelled the world everyday.

Datça needs growth to sustain itself whole year around. 

Is it possible to grow a seaside town’s economy in a sustainable fashion, by avoiding over-tourism? 

How would you engage with a partisan government who would pour money into an opposition town to improve its roads and infrastructure, without expecting a hefty return for itself?

On the upside Datça has a well educated population of Young Turks, entrepreneurs who share a strong ambition to maintain what Datça is known for, a lovely, clean, secluded, eco-friendly town. 

Streets are wide and clean. Recycling bins are available, though not consistently. Plastic poles are cleverly used to separate lanes. This works better than painted lane markers that are sometimes disregarded in Türkiye. 

On the downside, current economic hardship Türkiye is going through had an impact. It is August, but our hotel is operating at one third of its peak capacity. Recent mega price increases (e.g. petrol) caused all prices to spiral out of control, cutting back tourists, local and foreign alike. This is a recession and it is hurting both locals and tourists. People who visit Greek islands say “everything is cheaper and better quality over there”.

Other problems Datça is facing are long standing ones; bad roads, stray dogs, drivers disregarding pedestrians at crossings. 

The heat is unbearable. I think cossy Parisian coffee shops, small tables and small chairs are misfits here. 

I now move to a large breezy tea garden, protected by shades of thick tree canopies overlooking to an open plaza. 

Datça is breathing again.

Turkish References 

Sahil Kasabasında Yeni Bir Hayat, Datça’ya yerleşen 60 kişiyle söyleşi, Özgül Tuzcu, Ezgi Kurt 

• sayfa 30: paragraf: Datça nasıl değişiyor sence?
• sayfa 125: paragraf: Peki sorunlar yok mu?

Demise of wall clocks

In the foyer where the giant teddy bear with its lifeless gaze is watching crowds below, parents take pictures of their children with cellphones. Travellers from all over the world are buzzing about in the airport. This place feels like an extraterrestrial space station with air conditioners protecting humans from hostile desert weather outside. There is even a small botanic garden inside.

Battered by a fourteen hour flight, standing there before the final leg of our journey, we were happy regardless. We moved to an area where people watched an electronic display of flight schedules. 

I don’t wear a wristwatch. 

My cellphone was still showing Sydney time. I wanted to know what the local time was. 

I looked around, high ceilings, giant columns, gorgeous shop windows, shrines of Italian fashion brands, the giant teddy bear, and the flight schedule display, but no sign of a wall clock anywhere. 

Remembering gorgeous Swiss railway clocks and their imitations, I wonder what happened to them. 

We used to have wall clocks in foyers. They were useful artefacts of the analog age. If your mechanical wristwatch failed or you didn’t own one, you could always count on wall clocks. They were revered, communal objects.

With the rise of cellphones, we forgot wall clocks. 

Apple initially used the Swiss railway clock design without permission in iOS 6. Although the exact details of the licensing agreement are confidential. It was reported that Apple ultimately paid Swiss national rail operator SBB about CHF 20M (about US$ 22.4M as of January 2014) to license the use of the clock design. Apple later removed the design from its operating system with iOS 7 (Source: Wikipedia)

The cellphone clock is useful, precise, and doesn’t require readjusting. But cellphones are multifunctional devices, and displaying time is a demoted function. The clock is permanently present on the home screen and sometimes on the status bar. However availability of local time depends on the Location service that may not be available when you don’t have Internet access. That is often the case in airports.

My eyes scanned the flight display, no trace of the local time. 

Five minutes passed. Next to a label “Local Time” at the bottom of the flight display I saw a tiny digital clock. 

A deep sigh followed. 

The Infinite Conversation

The AI chatbot ChatGP is one of the fastest growing web platform. 100 million new subscribers per month. 

Giacomo Miceli, an Italian American computer scientist wrote an article in the Scientific American April 2023 issue.

In just few months using open source tools Miceli developed a website called The Infinite Conversation that enabled him to create a never ending fake conversation between film-maker Werner Herzog and philosopher Slavoj Žižek in their real life voices. 

Here it is, give it a whirl: https://infiniteconversation.com/

Just to re-iterate, this conversation was generated artificially by a computer. What you hear is not real.

Miceli mentions, in real life Herzog and Žižek often talk about philosophy and aesthetics. The chatbot versions were trained by their voice and with preference to their vocabulary. He adds, “Because of the esoteric nature of these topics, the listener can temporarily ignore the occasional nonsense that the model generates.”

Miceli asks audience not to take too seriously on what the chatbots are saying but imagine instead “the realistic-sounding speeches that could be used to tarnish the reputations of politicians, scam business leaders or simply distract people with misinformation that sounds like human-reported news.” 

Ankara 365 degrees from the eyes of an Aussie

• Kudos to Anatolian Civilisations Museum, a cultural sanctuary filled with a unique collection of rare prehistoric and ancient artefacts, curated to provide an engaging and educational experience. Things to improve: lights were too dimmed, it was hard to read most labels, and more alertness is needed by security staff to deter people from touching priceless artefacts with their greasy hands. For a reason unknown to me people love to take selfies with an Assyrian king than Eiffel Tower, putting their arm around lifeless stone shoulders of the king.
• Ankara Citadel and surrounding historical areas have improved significantly as touristic destinations compared to ten years ago. Most of the old houses have been restored, some have questionable authenticity though - I saw a plaster Russian bear sculpture (more like an arctic bear) in front of a hotel to attract Russian tourists, who were sadly nonexistent due to the war in Ukraine. 
• For foreign tourists taxis are reliable and recommended form of commuting in Ankara. Taking taxis from designated stops where they queue up next to a “taxi” sign is encouraged. Catching taxis casually may be less desirable as those drivers usually are freelancers who look and act like Mexican banditos. Their horses, or rather vehicles, may be less comfortable and less hygienic. 
• Most taxi doors will not hold when you open them, the door will close on your knee and crush it when you are getting out with your bag. This is probably part of the fun while visiting a Turkish metropolis. 
• In any case make sure the taxi driver resets the starting price before driving. Due to high inflation rate don’t be alarmed if the opening price in the afternoon is more expensive than in the morning, either way having a dispute with a taxi driver - any driver - is not recommended. 
• Taximeters are installed either on the mirror or on a box near the gear. If you are sitting next to the driver use your seatbelt. In the back, seatbelts are optional, the chances are they won’t even work. 
• Adherence to traffic rules on highways is good, apart from momentary speeding well above speed limits. Drivers mostly obey traffic lights in big junctions. Pedestrians who want to use crossings however should not assume anything and double check vehicles that may override rules. Apparently some drivers have peculiar fun from scaring people crossing. 
• Traffic in Ankara is chaotic in side streets, drivers - mostly taxi drivers - can be vulgar and aggressive, harassing people crossing the street, elderly and sick included (the fun doubles.) Pedestrians are second class citizens, just like slaves in Ancient Greek cities. Most drivers do not honour pedestrian crossings without traffic lights and some show needlessly aggressive behaviour towards pedestrians, as if they are Martians who attacked our planet.
• Most inner city pavements are poorly constructed. Walking becomes acrobatics around obstacles, sudden ends, parked vehicles, holes big enough to swallow bears and other mammals, invisible dents, hazardous material and dirt. 
• On some pavements you will find yellow coloured hard plastic strips in the middle along the pavement with circular or rectangular dents high enough to trip you - they are rumoured to be designed for blind by a sick person who doesn’t like disabled citizens. Blind people in Turkey are smarter than people who can see, they have to be, otherwise they would be dead. Therefore you wouldn’t see a single blind person using yellow strips designed for blind, in fact you don’t see them at all. Sometimes on pavements, scooters, even motorcycles suddenly appear out of nowhere. You need to be very careful while walking - you shouldn’t assume pavements are safe. You need to unlearn walking, a Homo Sapiens trait evolved 200,000 years ago.
• Ankara has air pollution problem even in summertime. A thick layer of exhaust fumes and dust hang over the bowl shaped city most days. 
• There are still too many smokers, smoking in public spaces. Pubs allow smoking in outdoor sections. Some restaurants and cafes have two sections for smokers and nonsmokers. If you are a non-smoker or a person suffering from bronchitis, asthma or related respiratory problems, you may be susceptible to unhealthy effects of passive smoking, pollution and dust. 
• Tap water is not drinkable unless you want to end up in the nearest hospital. For casual everyday consumption you need to buy water in pet bottles. It is very rare though you would find recycling bins for pet bottles. 
• There seems to be more visible plastic pollution than the developed world. Plastic bags are widely in use across all sorts of shops. Garbage collection bins on streets are filled with mixed trash including plastic, paper and metals. There are some bins designated to collect glass, but it seems people don’t use them. In the evening casual garbage sorters - rumoured to be Syrian refugees - come by with their carts, leaning inside the bins, legs in the air, tearing up bags and trashing around, in search for cans, bottles and paper. This manual process may be seen as effective part of recycling, providing income to disadvantaged but it is unhygienic causing dirt and unpleasant odour around bins. In general streets in Ankara are dirtier compared to Sydney.  
• Stray dogs are a problem. They are populous in fringe suburbs like Yıldız where they come from the wild and roam. From the plastic labels stapled on their ears, I gather some of these dogs are immunised by municipalities and casually looked after by public. Most of them look frail and hungry. It is quite distressing to see animals in this state. They move about in packs in cooler weather or at nighttime, at times in threatening ways. I was told they mauled two citizens walking in the Seymenler Park at night. So don’t assume they are harmless. 
• There are plenty of restaurants, cafes, pubs and eateries in side streets. Alcohol may not be sold or served within 100 meters of mosques - if you’re that near to a mosque you will suffer from sleep deprivation, so you wouldn’t want to drink anyway. 
• 10% tip is customary and recommended. I see it a sensible way to support economic hardship endured my majority. If I am really happy I’d go above 10%. 
• POS terminals are widely available, majority of the time they work reliably. As in any parts of the world, do not leave your credit card unattended and do not share or show your pin. You must be careful while using ATMs, as beggars or pickpockets can be wandering nearby, so as Kleenex tissue sellers. 
• Using cash in taxis is cheaper, they would overcharge, if you use POS.  There are foreign exchange shops with red signs  in crowded streets, they too provide good rate, not as good as credit card foreign exchange rate though. People who work in foreign exchange bureaus act like princes and princesses. They don’t smile, they don’t even look at you. Don’t get offended. They are like that.
• There are many coffee shops in Kavaklıdere district serving Italian coffee types, cappuccino, latte, espresso and the like. The quality of coffee served is poor most of the time. If you are an Aussie (the nation of coffee snobs) you will likely get frustrated. There are three problems, the coffee is not hot enough, they don’t know how to make good milk froth, and the coffee flavour left on your palate is so weak that you’d think you have covid-19. Cappuccino and latte are hopeless. The froth looks like carpet cleaning foam with a touch of baby shit, tastes like sour canned milk powder from 1945, mixed with a touch of gunpowder, found in a Nazi bunker in Normandy - I am telling you it will really put you off, so much so that you will wonder if you are teleported to a different planet. Your best chance of having a decent cup of coffee is by ordering espresso or Turkish coffee, even so the chances are fifty fifty. 
• In cafes, pubs and restaurants you will be greeted by an army of waiters. There are so many waiters that you feel ambushed by an endless North Korean army division - as soon as you tackle one, two more appear from the bush, jumping on your neck with a shriek in Korean. Waiters watch you all the time, basically checking if your plate or glass is empty, and they walk around to attack you as soon as they see an opportunity - when your plate or glass is empty. You need to think strategically and find a table and a sitting orientation to minimise waiter harassment, otherwise instead of eating or drinking properly by your mouth, you will feel like food and drinks shoved in the wrong end. Ironically Turkish waiters disappear when you need them most, and that is when you want to ask the bill. 
• In contrast waiters in Australia are a mystery. They exist in spirit - they are completely invisible. Even if in rare cases you see them, they demand a ceremony equivalent to the King of England’s coronation. They do everything in their power to avoid eye contact, just like the King of England. 
• In summertime some restaurants, pubs and cafes open outdoors sections with limited air flow due to shades and partitions around the area. If you sit in the middle or at the back, you will be smoked to death like cockroaches, as people chain smoke just like Humphrey Bogart did in Casablanca. If you sit at the pavement edge, you will be harassed by beggars and Kleenex sellers. Don’t be tempted to pay them. They are professionals, experts in exploiting your sense of pity - they probably make better money than waiters most of whom are university graduates on minimum wage.