Ben Hamner, Kaggle co-founder and CTO, held a Quora Session last month answering questions on the future of Kaggle, machine learning and AI, and data science workflows. Here we highlight his advice for studying machine learning in eight steps.
Now is better than ever before to start studying machine learning and artificial intelligence. The field has evolved rapidly and grown tremendously in recent years. Experts have released and polished high quality open source software tools and libraries. New online courses and blog posts emerge every day. Machine learning has driven billions of dollars in revenue across industries, enabling unparalleled resources and enormous job opportunities.
This also means getting started can be a bit overwhelming. Here’s how I’d approach it. If you get stuck anywhere in this process, searching Kaggle (there’s a good chance someone’s hit your issue before) and posting on our forums (in case someone hasn’t) is a great way to get pointers and get unstuck.
1. Pick a problem you’re interested in
Starting with a problem you want to solve makes it a lot easier to stay focused and motivated to learn, instead of starting with an intimidating, disconnected list of topics (you’re a Google search away from many of lists of machine learning resources, I’m not providing another one here). Solving a problem also forces you to deeply engage with machine learning, instead of passively reading about it.
Solving a problem ... forces you to deeply engage with machine learning, instead of passively reading about it.
Good problems to start with have several criteria:
- They cover an area you’re personally interested in
- Data is readily available that’s well-suited to addressing the problem (otherwise the bulk of your time will go here)
- You can work with the data (or some relevant subset of it) comfortably on a single machine
- Don’t have a problem that comes to mind? No worries! We provide a nice onramp of machine learning problems at Kaggle through our getting started competition series. Start off on the Titanic competition.
2. Make a quick, dirty, hacky, end-to-end solution to your problem
It’s really easy to get bogged down in one implementation detail or carefully tuning the wrong machine learning algorithm. You want to avoid this.
Your goal here is to get something super basic in place as quickly as possible that covers the end-to-end problem, from reading in the data, processing it into a form suitable for machine learning, training a basic model, creating a result, and evaluating its performance.
3. Evolve and improve your initial solution
Now that you have a functional baseline, it’s time to get creative. Try improving each component of your initial solution, and measure the impact to see where it makes sense to spend time. Many times acquiring more data or improving data cleaning and preprocessing steps have a higher ROI than optimizing the machine learning models themselves.
Part of this step should include being hands-on with the data - inspecting individual rows and visualizing distributions to have a better understanding of its structure and oddities.
4. Write up and share your solution
The best way to get feedback on your solution is to write it up and share it. Writing about your solution mean you’ll engage with it in a new way and understand it better. This enables others to understand what you’ve done and provide feedback, helping you learn. It also kickstarts your machine learning portfolio, which will help showcase your abilities and get a job.
Kaggle Datasets and Kaggle Kernels are an effective way to share your data and solution, get feedback from others, and also see how others extend your problem. This starts fleshing out your Kaggle profile as well.
Share your project as a dataset on Kaggle where you can publish kernels (notebooks/scripts) and get feedback from the community.
5. Repeat #1-4 across a diverse set of problems
Now that you’ve done this for a single problem that you’re interested in, do this several more times across a different set of domains.
Did you start off with tabular data? Work on a problem that involves less structured text, and another that solves images.
Was the machine learning problem structured for you initially? A lot of the creative and valuable work is figuring out how to go from a loosely-defined business or research objective to a well-defined machine learning problem in the first place. Work through this for one problem type.
Kaggle Competitions and Kaggle Datasets provide a good starting point for both well-defined machine learning problems and raw data sources that’d be suitable for machine learning.
6. Seriously compete in a Kaggle competition (if you’ve not already done so)
Giving your best shot at the same problem that thousands of others are hard at work on is a tremendous learning opportunity: it forces you to iterate on the problem over and over again, and then exposes you to what works effectively on the problem.
The forums for an individual competition are a rich resource on how others are approaching it and debugging issues with your approach, kernels provide exploratory insights about the data along with an easy way to get started on a problem, and the winning blog posts at the end showcase what ultimately worked best.
Kaggle Competitions also provide a unique opportunity to team up with others. Our community has a diverse set of background and skills, so everyone has something to teach and something to learn. You never know, you may meet your future colleague on Kaggle!
7. Apply machine learning professionally
This enables you to spend most of your time on machine learning and really helps you level up. Deciding on the type of role you’d like to pursue and building a personal portfolio of projects related to this is a strong starting point. If you’re not ready to start interviewing for machine learning positions, then taking on new projects in your current role, seeking consulting opportunities, and getting involved with civic hackathons and data-related community service opportunities are additional ways to get a foothold. Professional work often requires and is greatly enhanced by strong programming abilities - improving this with focused projects yields many downstream payoffs.
Valuable opportunities for professional machine learning work include:
- Applying machine learning in production systems
- Focusing on machine learning research and pushing the state of the art forward
- Leveraging machine learning in exploratory analyses to improve product and business decisions
8. Help teach others about machine learning
Teaching others will solidify your grasp on the core concepts. There’s a lot of different ways to go about this - pick the ones best suited for your style. They include:
- Writing research papers
- Giving talks
- Writing blog posts and tutorials
- Answering questions on Kaggle, Quora, and other sites
- In-person mentorship and tutoring
- Sharing code examples (in Kaggle Kernels and on GitHub)
- Teaching a class
- Writing a book
Check out all of Ben's top answers from his Quora session here. Plus, Quora's Duplicate Question Pairs competition is the perfect challenge for newcomers to Kaggle Competitions.
