Three article summaries, which I’m breaking out from my usual “what I read” posts because of the common theme (of CS research & grad school) that connects them.
An Opinionated Guide to ML Research
Author: John Schulman
How I found this: /r/machinelearning
Summary: Schulman is a research scientist and founding member of OpenAI. This post is advice to aspiring researchers, and his keys to success are “working on the right problems, making continual progress on them, and achieving continual personal growth.”
- Working on the right problems: this is primarily advice on how to pick the right problems. Read a lot of papers (critically!), work in a group, talk to experienced researchers, and spend time reflecting on what is useful. He also discusses the differences between idea-driven and goal-driven research.
- Making continual progress: this is advice on developing a long-term habit of problem solving. He suggests keeping a lab notebook with ideas, results, and what was worked on, and reviewing it periodically. He addresses the question of when to switch problems, and that in his experience switching too frequently is more often a problem than not switching enough.
- Personal development: the author recommends reading textbooks (even outside of school, because they cover topics broadly) and PhD theses (which usually have lots of background information), along with seminal papers to understand the frontiers of the field. The “less exceptional” papers are also worth keeping an eye on to understand what ideas are being tried out.
Thoughts: there’s a lot of great advice packed in here. Keeping a lab notebook has been extremely helpful to me in helping me remember what I’ve worked on and what ideas I have (or had ages ago). I like the strategy of reviewing the notebook into a summary every couple of weeks, since I feel like this would help me to maintain a coherent research direction.
The advice on switching problems also hits close to me. While my problems at work are defined by my team and the business, the methods I use aren’t; and I can think of a couple of occasions when I was too quick to abandon a method that still showed a little bit of promise. I’m trying now to be more intentional about what ideas I try out and what I leave for later.
Advice to (prospective) grad students
Author: Vivek Haldar
How I found this: no idea anymore
Summary: this is long post of advice to prospective CS PhDs (oh hello!). Some of the highlights are:
- Be absolutely certain that you want a PhD. For going into academia, the odds are stacked against you. For going into industry, the financial opportunity cost is almost certainly not worth it when considering the 5+ years of missing a salary. “This is what I like to call a clamped decision. If you are not fully convinced that the answer is “yes”, then the answer is “no.””
- Choosing where to go is largely a function of happiness & quality of life + the learning & research you do. These, in turn, almost entirely depend on your advisor. Ask the current grad students who work with possible advisors. Oh, and location matters too.
- On why to do a PhD: “Like I said above, this is a question of self knowledge. The circular answer is that you should do it if you are the type of person who would enjoy it.”
Thoughts: this is good advice, and mostly consistent with what I’ve heard before. I have done a great deal of self-reflection about whether or not grad school is for me, and I anticipate a lot more in the coming year. I think the biggest questions for me will be (1) do I want to devote 5+ years of my life to this, (2) can I pursue the opportunities I want right now (i.e., as a data scientist in industry), and (3) how much will this affect my personal life, relationships, and location. These are all worthy of deep thought.
What is life like for PhDs in computer science who go into industry?
Author: Vivek Haldar
How I found this same blog as above
Summary: this is a series of short answers to questions about what it’s like to be a CS PhD in industry (the author works at Google). The author opens with a discussion about the “industry research” model that was popularized by Google. Some of the answers are not PhD-specific (e.g., “how much mentoring is there” or how work-life balance is). The answer to “are you only qualified to work in the subfield you studied” is a resounding “no,” which is consistent with my experiences at Nielsen.
Thoughts: this honestly doesn’t have that much new information for me. I work with a lot of PhDs at my current job (my manager, one of two teammates, and many people who sit near me on other teams). In many ways, I’m doing the same things that they are. The author says as much–“you will usually do the same type of work”–so the real question is whether this is really what I want to do.