Deep learning

By Eric Collins

Professor Jeff Hong is Endowed Chair Professor in the Department of Economics and Finance and Department of Management Sciences. This semester he is offering a voluntary study group with the theme deep learning. The group follows a Stanford University online course “CS231n: Convolutional Neural Networks for Visual Recognition” and consists of students at all levels from undergraduate to PhD, as well as faculty members. City Business Magazine went along to one of the seminars to find out more and talked with Kiko Zhang, MPhil, Management Sciences, Yifan Zhang, Research Assistant, and Dr Zhankun Sun, Assistant Professor in the Department of Management Sciences.

Why are you offering this study group?

Jeff: I want to learn. Machine learning has changed many things. Back when I did my PhD study we didn’t have these tools. I also want my PhD students to learn. I don’t know if they are volunteers of this group in the true sense of the word! In the management sciences field we can see that machine learning and data science are changing the way we do everything. Now, when people talk about this it is no longer a mystery.

How important is it for PhD students to learn tools?

Jeff: I think it is very important. From my own experience as a PhD student, I took more than 70 different courses. I didn’t know how I was going to apply them. I was just curious. It’s like Steve Jobs said, if he hadn’t taken a calligraphy course at university there wouldn’t be all the fonts in programmes today! Even if our current research is not directly linked to this, give me some time. I think in two or three years, I may write papers using these tools.

How does deep learning work?

Kiko: If we are talking about visual recognition, you have this ground truth label with say 10 classes of image and you calculate the difference between the ground truth and a value x. But this method can be applied to any prediction problem. So, with the price of a flat you have some trigger parameters such as the location of the house, number of rooms, maybe the wealth of the family. Prior to deep learning you wouldn’t have a hidden layer, but now from just these three values, you can try and predict the number of family members, the school district, the distance you have to walk to school. Ultimately you predict the house price.

Where can deep learning be used?

Kiko: The techniques can be applied to any research area, for example, engineering, image recognition, or finance – to any classical statistical prediction problem. The main motivation in management sciences will be reducing costs.

From the research perspective you can use Python and all the APIs it is built on in any field. If you are in genetic science you can predict gene mutations. In medical science you could also use the tool.

Can you give a specific example?

Yifen: Take bicycle sharing. Ofo has flooded cities with bicycles, but sometimes you still can’t find one. The company doesn’t have enough data to train the neural network to get the bicycles where they are needed.

Now we have many neural networks, and need a lot of data to train these networks and make them more efficient in solving problems. We often don’t have enough data to solve our problems. So there is a tool called Generative Adversarial Networks. This is a class of artificial intelligence algorithms used in unsupervised machine learning, implemented by a system of two neural networks contesting with each other in a zero-sum game framework. They can generate more data to train our neural networks.

Is this study group helpful to you?

Yifen: It’s been very beneficial. I now know how to run a programme on Google Cloud. You can also initiate your Graphics Processing Unit instance, you might have millions of parameters.

Kiko: It’s my first time working with Python programming language, and very beneficial. Now when I read a paper from deep learning, I am no longer lost!

Who is teaching the class at Stanford?

Jeff: This class is being taught by a PhD student for Stanford undergraduate students (something that would not currently be possible at CityU), and they are teaching research papers that are appearing in the same year. That’s completely cutting edge and it’s the first time I have seen that.

Which classes were important to you in your own PhD study?

Jeff: As I said I took dozens of courses. One on Computer Aided Design stands out. Half the course was talking about computational geometry, which had nothing to do with my field of study. However, we learned this thing called Voronoi diagram, a partitioning of a plane into regions based on distance to points in a specific subset of the plane. I spent a long time writing a computer programme. I never expected that one day I would use Voronoi to solve what turned out to be the core problem (a simulation optimization problem) in my PhD. People said, how come you got so creative? But it wasn’t like that. One day I just thought “Oh, this is very similar to the Voronoi diagram.”

Steve Jobs said, life is about connecting dots backwards. If those dots have been planted, they will allow my students to move forward.

Deep learning (also known as deep structured learning or hierarchical learning) is part of a broader family of machine learning methods based on learning data representations, as opposed to task-specific algorithms. Learning can be supervised, semi-supervised or unsupervised.

Deep learning models are loosely related to information processing and communication patterns in a biological nervous system, such as neural coding that attempts to define a relationship between various stimuli and associated neuronal responses in the brain.

Deep learning architectures such as deep neural networks, deep belief networks and recurrent neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics and drug design, where they have produced results comparable to and in some cases superior to human experts. ––Wikipedia

Should PhD students take broad study programmes?

Jeff: PhD students often concentrate on a particular research area, but I want my students to be trained in a broad sense. Research topics come and go. If you only learn the topic of the day, five years down the road, you may become very narrow and may fossilize. You won’t be able to solve new problems because you don’t have the tools. That’s why I am still learning new tools!

I think the relationship between advisor and advisee is quite important. To some extent the way my students think after they have finished their PhD is affected by the way I think. For myself, I see similarities between the way I think and my advisor, and his advisor too. In a sense it is a learning lineage that we are passing down. As Isaac Newton said, we stand on the shoulder of giants. We use the tools that we have to solve problems. And in the process, we may invent new tools.

Can this study group experience affect current teaching?

Zhankun: I agree with Jeff. I am not too far away from being a student myself, and I hear about new methods, and wonder if I can solve problems using them. Learning by yourself is not always the most effective way. Your professors are better placed to pick out what is the core content of a particular subject. Now, machine learning is no longer so mysterious for me.

First this study group satisfies my own curiosity, second it might be helpful in my future research, you never know. It’s only afterwards you can make the link. Lastly, it will have an effect on my own teaching. I am teaching a course which is quite statistically related, but there is a lot of classical statistical content, and a lot of it was invented 100 years ago. Image recognition is being applied to the driving of autonomous vehicles now. So, this is an opportunity to introduce cutting edge content to my courses.

Machine Learning for Business Research

AI is everywhere. Face detection, speech recognition, predictive advertising, email spam detection, are all AI driven. Machine learning is equally important in business research, and is enabling research work of unprecedented scope. The new learning tools deliver us new data and methods, and also focus on new questions such as out-of-sample prediction and high-dimensional algorithms.

Machine Learning for Business Research, FB8918, is a new elective course to be offered by the College of Business from Semester A 2018/19. Machine learning stands at the core of many business models nowadays and this course will teach machine learning models and tools and enable students to conduct related business research. The course will cover supervised learning in depth, including regression, classification, regularization, tree-based methods, ensemble methods etc., and will also introduce the basic concepts and tools in unsupervised learning, including clustering and principle component analysis, etc.

The course focuses on practical training using business data including marketing and financial market data, as well as unstructured text data in news media. Applications with business data use clustering, tree & forest, support vector machines, boosting and ensemble methods, and neural networks. These include classification with applications in marketing data, prediction problems and applications in finance, and text analysis and applications in event study.