7 ms·

We Need a Yelp for Doctoral Programs

180 points by etattva 6y ago

westurner 6y ago
How are the data needs for such a doctoral and post-doctoral evaluation program different from the data needs for https://collegescorecard.ed.gov https://collegescorecard.ed.gov ? Data: https://collegescorecard.ed.gov/data/ https://collegescorecard.ed.gov/data/ Data documentation: https://collegescorecard.ed.gov/data/documentation/ https://collegescorecard.ed.gov/data/documentation/
- sgillen 6y ago
  Reviews seem like the biggest need. “The culture in the X department is terrible, everyone works 80 hours a week and is miserable” “Night life is great at least, the grad students usually go out for student nights at blah blah after seminar” “Underrated X program because at university Y, don’t be fooled we literally have 4 of the 5 top researchers in field Z” Etc.
  - etattva 6y ago
    But it is so hard for people to give reviews. People seem to give good review when are they are very pissed off or very happy. Later seems to be a very rare case.
    - p1esk 6y ago
      I couldn’t make sense of your comment.
      - liammonahan 6y ago
        I think they mean “good” in the sense of being helpful reviews: positive or negative.
    - perl4ever 6y ago
      I have always thought that bad reviews are sufficient for all purposes, and the only reason to even have good reviews is so the reviewees don't feel overly persecuted. Any product or service will have occasional customers who are either unreasonable or have bad luck. If you have a lot of bad reviews, then as a prospective customer you want to see if all of them fall under the two categories or whether there is a pattern that is relevant to you. It seems to me that a review database works fine even when all the reviews are bad, and the only regulation it really needs is an effort to prevent one single individual with a vendetta from overwhelming it.
etattva 6y ago
collegescorecard measures the outcome. In the post, Profesor Leonard talks about doctoral programs being rated based on research outcomes.
michaelmrose 6y ago
So someone to extort colleges to remove bad reviews?
- etattva 6y ago
  Ha ha I know where you are going :) My take from the post was that there needs to be some information provided to end users based on which they should be able to decide what colleges to choose. Ranking thatis dictated by third parties cannot be a criteria. And student should not spend too much time emalining each and every college they are interested in.
  - deleted 6y ago
    [deleted]
eli_gottlieb 6y ago
No. We don't need a consumer review website, because PhD students (I am one) are not the consumers of doctoral programs. We're the workers. We need a union. If we're in a good program, we're receiving valuable training both in our coursework (or before PhD in an MS/MA if your country does it that way) and in our research work. Ideally, we're building up a network of collaborations and acquaintances in the scientific community that gives us an "in" for the next stage of our career, while also being mentored by world-class experts in our fields. In a not so good program, we're used as TAs for a few years and then rapidly encouraged to drop out. Either way, most of what we do is not like most of what an undergraduate student does. It is like what an entry-level professional does.
- mxcrossb 6y ago
  Yeah what we really need is a glassdoor equivalent.
- musicale 6y ago
  > PhD students (I am one) are not the consumers of doctoral programs. We're the workers I wish I had the reference, but I remember reading about a study which showed (unsurprisingly) that the high research productivity per dollar in the US is primarily due to the below-market salaries of graduate research assistants and postdocs. Not so much capitalism - more like a guild system with a lengthy (and perhaps semi-permanent for many postdocs and adjuncts) period of apprenticeship and indenture.
- chrisseaton 6y ago
  On a PhD you work for yourself - you shouldn’t need a union against yourself!
  - kayaeb 6y ago
    Really depends on your mentor. I've seen some gross abuses where the union was absolutely essential. Also the union hooked up like BBQs and whatever so like, that's nice.
    - 6y ago
aspaceman 6y ago
This article really rubs me the wrong way. I don't think I would find a Yelp very valuable, and the most important metric for a PhD program isn't job placement numbers. The idea that job placement correlates to PhD quality is just plain wrong. Incredibly intelligent students leave PhD programs through no fault of their own, and people who get nothing out of a PhD program can get a cushy job easily. It's a horrible metric. I found that the difference between faculty members at the same institution can be immense. This is because faculty have great control over your life due to how doctoral funding works in the United States. If you wish to continue a Biology degree, you must be willing to work with this faculty member, in this area, on this specific problem. Some faculty are rigid about a specific problem, some are not. Even more complicated is that some students need more guidance than others. Or some students want much less. And the student doesn't necessarily know what they need before entering the program. This is why I think it's smarter to give doctoral students more freedom and flexibility in switching faculty. The school should be supportive of a student in this process rather than focusing on their job placement numbers.
- kyboren 6y ago
  > I found that the difference between faculty members at the same institution can be immense. Absolutely! > This is why I think it's smarter to give doctoral students more freedom and flexibility in switching faculty. The school should be supportive of a student in this process. This is important, but it's not enough. The nature of a PhD means that switching faculty is often not an option. Students 3 years into their thesis work with a budding publication record on a topic often can't realistically just switch to another professor, even if everybody is on board with it. In most cases that means starting over from scratch with a new topic. So instead students will grind out a final couple (or more) miserable years in order to finally get their degree.
- foobarian 6y ago
  I agree with you, and further think it would be awesome if each doctoral program offered "user manuals" for each potential advisor in the department. Things like, "Professor X is very hands off, so you'll be able to do a lot on your own, but make sure you get on his calendar to get advice." Or, "Professor Y is a slave-driving task master but very capable; so make sure to stand up for yourself or you'll get crushed." Of course this is tongue in cheek and would probably never get formalized. I got this kind of advice from people who already finished their PhDs while I spent a year in industry before grad school. Not everyone can do this but maybe there is some other way to cold-contact finished PhDs and see what they know about the faculty you're considering.
deleted 6y ago
[deleted]
deleted 6y ago
[deleted]
andrewacove 6y ago
Even if you have a terrible experience in your doctoral program, much of its outward value is in the program's reputation. Giving a program a negative review hurts the perceived value of your own education.
r-w 6y ago
I think Glassdoor[1] would be a better analogue. [1] https://www.glassdoor.com/about-us/ https://www.glassdoor.com/about-us/
nabaraz 6y ago
A decent review site would be awesome. I have been looking at WGU for an online MBA degree. Reviews are just scattered everywhere from Reddit to affiliates websites.
- bruceb 6y ago
  Can I ask why WGU? Instead of at least a mid-tier school ?
  - OldHand2018 6y ago
    WGU is accredited. If you are sufficiently motivated, it is fast and cheap. It doesn't require you to move or to quit your existing job. I can't speak to the motivations of the person considering it, but perhaps s/he only needs to check a "do you have an MBA?" box on an application to be able to move forward in their career.
  - nabaraz 6y ago
    WGU is a competency based school. So I can finish my MBA from anywhere between 6 months - 3 years. On brick-mortar schools, you are limited to 3-4 classes per semester.
ISL 6y ago
Good luck getting honest reviews. Graduates and recent post-docs are dependent upon good letters of recommendation from their advisors for perhaps a decade after leaving their research groups. The truly disillusioned won't mind complaining, but those with even constructive criticism will feel constrained in their ability to speak out and be specific. The graduate-student experience is deeply advisor-specific. Professor A may be exploitative, while Professor B may fight tooth-and-nail for student success. The statistics are low, too. Most professors will only graduate a few students per decade.
- musicale 6y ago
  This is a good point. Anonymous reviews are not possible with such a small and easily identifiable set of reviewers.
- dopu 6y ago
  Completely agree. I've become increasingly convinced that we'll need a fundamental restructuring of the academy if we want to fix any of these issues. The fact that places like MIT or Stanford will hire professors simply because they output Nature papers every year, regardless of the fact that they are totally abusive to their trainees, is a perfect demonstration of it.
  - lumost 6y ago
    Doesn't this explicitly favor the high-output at any cost professor? A professor who had a high nature output could attract more PhD students and structure their program such that they take more of the credit than you would expect. This wouldn't work in all fields, but there are quite a few disciplines where the biggest barrier to research is having funding and someone to do the work.
- jseliger 6y ago
  In many disciplines, too, the answer is "don't go to any program." Most humanities fields, for example, have numerous essays written about why doing grad school in them is a poor life choice, like my contribution: https://jakeseliger.com/2012/05/22/what-you-should-know-before-you-start-grad-school-in-english-literature-the-economic-financial-and-opportunity-costs/ https://jakeseliger.com/2012/05/22/what-you-should-know-befo.... One could say the same of most social sciences.
ramraj07 6y ago
We dont need a Yelp for doctoral programs, we need a Yelp for research labs! It's more about the lab you join than the program itself that makes or breaks your life. In fact, I have heard that minimally there's kind of a toxic labs blacklist maintained in some Chinese message boards. At least that's what a fellow postdoc told me when I mused why we suddenly stopped getting applicants from China to our lab!
- frandroid 6y ago
  There are PhD programs not attached to research labs. I would say the majority of doctoral students are not attached to one.
  - mmhsieh 6y ago
    by research lab, he means a research group led by a specific faculty member in an academic department. in other words: he is talking about reviews on specific professors as doctoral degree advisers.
- chrisseaton 6y ago
  More that the lab you’re in it’s about your specific advisor. Two advisors in the same lab could work in a completely different way.
  - joshvm 6y ago
    If you have an issue, the onus is on the institution to fix the problem. It doesn't matter if you're part of a lab or not, your advisor has a boss, and your university should have a support network. If it doesn't, then strongly consider going somewhere else. It also totally depends on you. I understand that in the past a lot of people clashed with my PhD supervisor, but I never had any issues as a student (and we got on well). What you need to ask is: does the instutition side with the academic in cases of misconduct (e.g. inappropriate behaviour?), or do they side with the student? Do they have a robust pastoral program? Would the students say that they feel supported? I can categorically say there are awful advisors in all institutions. A friend got a position in Cambridge. While their advisor was on vacation my friend - innocently - set up a group Slack. When the advisor returned, friend was accused of trying to steal the group. It was a bizarre situation. I know people at less well-known universities with more mundane issues, like not being supervised, and it's taken them years to get the department to do anything.
glial 6y ago
It would be better to just contact current students at the lab and offer them coffee for a conversation.
- CarbyAu 6y ago
  While I agree with the sentiment, many potential students are looking overseas. Is there an Internet equivalent of "coffee and conversation"? Steam game gift and discord maybe...
  - henrikeh 6y ago
    You don’t have to bribe them... you can just write to them.
    - CarbyAu 6y ago
      A lot of people will say things they won't write, especially in a nice environment.
      - henrikeh 6y ago
        The schedule a call when you found someone you can talk to. Of course you lose something when physical presence is impossible, but in my experience you can always find someone who is happy to write, chat or call.
crb002 6y ago
The last time I tried to interact with Iowa State University ECprE, a professor who I won't name asked me for a $20,000 bribe to do research with their lab. Confirmed their culture is just as toxic as it was a decade ago.
- ausbah 6y ago
  why not nane and shame?
abakker 6y ago
We barely need a Yelp for restaurants. I think it is time for the internet to acknowledge that the pointless drivel in star rating systems is just not helping anyone. If you’re going to start a system for rating things, give it more nuance and value.
- sn41 6y ago
  Yup. Recently bought an affordable mechanical keyboard without consulting any review websites. Am very happy with it for about a month. Then I went to Amazon to see the reviews, out of curiosity. They are very negative for the product I am very satisfied with. Had I done due diligence, I would have lost a bunch of cash for a mere marginal improvement which none but connoisseurs can discern. I think there are a few things in life we must decide carefully - like a partner, which doctor to consult etc. Reading reviews for most of the other stuff is overthinking, and pointless.
- fastball 6y ago
  I actually have a plan for this! I want to build a ratings website where you rate things in the same way they generally structure psychological tests, by responding to affirmative statements with something between "strongly agree" and "strongly disagree". As an example, if you were rating films, questions could be: "This film was enjoyable." "This film had an engaging plot." "The acting was believable / appropriate for the film." etc. Ideally you'd have about 5 standard statements that you respond to for a particular thing you want to rate (whether its films, books, food, or a university course). Better than a star rating system, because the problem with stars is that you end up with scores that always hover from 5-8 (if out of 10) in a very not useful way (see IMDB). It's also better than what RT / Netflix / many others have moved to (just voting with up/down) as those require a lot of data in order to really give interesting results, where as with this system you can start to find interesting insights almost immediately, or within a single person's own collection.
  - jkinudsjknds 6y ago
    People not answering questions correctly is precisely the reason many companies switch to thumbs up thumbs down. The precision of the measurement just isn't there for good / bad. Collaborative filtering with just these simple metrics works decently well, the issue is most sites like netflix have difficult UIs to actually use it as a poweruser. The average consensus of people on how good a thing is on some sort of scale, is pretty worthless. The consensus of people who seem to be like you on previous works is kind of useful. Strongly disagree, disagree, neutral, agree, strongly agree is just a 5 star system with a different name.
aleister_777 6y ago
This is hilarious. These people have contributed nothing to society, else they wouldn't required a medium to push their ideas. Your PHD means literally nothing of value!!
dhosek 6y ago
Basic rule: if you're paying for a doctorate (aside from a professional degree, like an MD or JD), it's a bad idea.
dhosek 6y ago
I've done my best to create a ranking of graduate creative writing programs by measuring the appearance of programs' graduates in the top prize anthologies. It's been generally well received although there has been some resistance to the idea of ranking programs like this. There's still a lot of work to be done on it, but the basic structure, at least is in place. http://creativewritingmfa.info/rankings/ http://creativewritingmfa.info/rankings/
- etattva 6y ago
  nice work. How do you keep it udpated?
  - dhosek 6y ago
    There's a fair amount of manual data collection. I get an excuse to enter into e-mail correspondence with 100s of writers each year, although I have most of the initial contact programmatically driven. There's also automatic annual e-mails of program directors in August to get the latest data from them and for the top schools on the list, I'll make calls if they don't answer the e-mails. In the fall I end up spending an hour or so a day dealing with all the e-mails and data entry.
blueboo 6y ago
The problem isn’t that we’ve locked opinion inside communities. It’s that doctoral programs are so poorly run. That universities are waiting for quantitative data to instill action is more sign of dysfunction. Here’s yelp for grad programs. Write down the name of the program you’re looking at and make an asterisk. One star — it’s already more accurate than Yelp on restaurants.
vnorilo 6y ago
I am not sure a Yelp-like system would work for the reasons others have mentioned. I would suggest these metrics to help you evaluate a doc program from the outside: Do early-stage grad students get (any) authorship credits together with professors or post docs? (Getting started) Do they eventually get first author status? (Independence) Do students go to conferences and read papers? Or does a prof just go and read all 10 from her institution? (Networking) Do fresh doctors land post docs, tenure, or good private sector jobs? (Outcome) A lot of this can be deduced by google (scholar) and scanning conference proceedings and video recordings. (defended my doc in 2016 in Finland, been fairly succesful at post doc funding as well as private sector)
- kayaeb 6y ago
  protip: if your prof is tenured and still first author on more than 2 papers a year, they're siphoning their student's work, or they're setting you up for pipelining positions (which are not horrible, but, be aware that's what's up). I've never had a prof insert my work into their own papers, they've hooked me up to contribute to other teams, but with us it was always "my" project. If you get a chance to see them present at a conference (lots of these are on video online nowadays), check if they specifically mention their students in the presentation, that's a green flag.
  - vnorilo 6y ago
    Good observation. Writing as second author was a very positive factor very early on. I was an assistant to a research fellow from my second year at uni, mostly because of coding skills. Pretty soon they encouraged me to write a pragraph or two about the implementation details, and I got added to the author list. I felt that was appropriate and a great kickstart to a research career (got my first MIT press journal credit before my bachelor) I agree however that professors growth-hacking their publication lists with student labor is problematic indeed and looking at first/second author creds is a good way to flag such behavior.
bosswipe 6y ago
There's a simple objective metric that could capture most of what you'd want to know about any school and put USNews's rankings out of business: graduates' achievement. How much are graduates making 10 years later? How many have prestigious jobs or other accomplishments such as research grants or awards? Universities do track these kinds of metrics as part of their alumni fundraising efforts but they're rarely publicly disclosed.
- michaelt 6y ago
  > most of what you'd want to know about any school [...] How much are graduates making 10 years later? Among my friends in graduate school: * One was told by their adviser they had enough for a thesis in their second year, and found time to organise a university-wide Dance Dance Revolution championship. * One was shouted at by their adviser for not coming in to tend to their experiments on Christmas day - the first day they'd taken off that year. I think a prospective student would like to know which of those experiences they're in store for, regardless of future earnings. Indeed, if you want to maximise your salary, I'm not sure you should go to graduate school at all...
tomashertus 6y ago
I was thinking the other day that I need a Yelp-like experience for technical tutorials and creators of educational content online. I'm sure I'm not the one who ran into so many shitty tutorials/how-to guides/videos through the years.
supernova87a 6y ago
The problem here is that a Yelp rating system for restaurants can be consumed by anyone because everyone wants a good-tasting, cheap, pleasantly-served meal. There's not a lot of range of desire and recipient goals. Grad students on the other hand, come with a huge range of: -- Motivations for doing a doctorate (say, purely for job prep, versus wanting a life of academia and intellectual fulfillment), -- Skills and knowledge (some already essentially begun their doctoral work and have achievements already, versus those who will be wandering around searching for a topic for years), -- Tolerance or willingness to engage in ambiguous, possibly dead-end leading places with no clear sign of success, -- Need for financial reward / subsistence and productive return for time or opportunity cost traded off. Add to that in many fields, the supervising professor is the greatest variable, more than the program or university -- and any rating system will be irrelevant or worse, misguiding. One person's paradise could be another's hell. Unless the rating is so dumbed down to generic factors in which case what are you really informing people about? To echo some advice of a contrarian grad school dean, go into grad school because you know what you want to write about, who you want to work with, and what you want to come out the other end with. Otherwise, the bulk majority of people going into grad school "on autopilot" or based on a Yelp rating are going to be sorely disappointed at some point.
electro_blah 6y ago
LOL You clear have no idea how Yelp works. Yelp employees and their friend will leave you bad reviews and ask you to pay (extort) in order to remove/hide them. Louis Rossmann (Search YouTube) has proof.
catsarebetter 6y ago
Doctoral programs are at least 4-5 years long and crazy emotionally challenging as well as a pretty surprising drop out rate, if someone doesn't take time to research them deeply before they commit to something like that then I think that person needs to learn the lesson that that will teach them. Yelp is the surface level easy way. Networking and cold-calling tons of current phd and master's students over the course of undergrad and during internships is how everyone does it.
ganfortran 6y ago
For real. But the title itself is something worth a moment of reflection. Doctoral programs are supposed to be scarce, selective and of high profile. Now they are being treated as a service that can be requested and compared against.
- kayaeb 6y ago
  This really rubbed me the wrong way, the closing sentence sums it up: "What if a doctoral program’s prestige arose, in part, from the way that it treated its students? We should dare to dream of such a thing." There is a huge problem in academia where the number of PhDs is growing exponentially, and the number of positions is staying basically flat. I strongly disagree with this "participation award" mentality of "you deserve it." It's... leading a lot of people to make really stupid career decisions and I think a lot of people are doing it just to get this gilded club to bludgeon people who don't have PhDs -- that's the only reason I can think of for a person to hope for an "easy" PhD experience. My advisor being rough at the right times (and in good spirit) was absolutely essential to the defense, which is essentially gladiatorial combat against your father (well, mother in my case), when both you and your advisor hope you can defeat the master in your specific realm. I would not have been able to do it if my advisor pranced about making everything pleasant and easy and convenient.
noelwelsh 6y ago
Any time education is discussed on HN it should come with a mandatory notice: - Your experience is no necessarily indicative of others' experiences - The US education system is not indicative of education systems in the rest of the world.
hymnsfm 6y ago
Not a fan of Yelp, but I see a need for a Yelp-like lookup for landlords and superintendents.
Upvoter33 6y ago
The problem in doing this is that saying "school X is great for getting a Ph.D." will never likely generally be true, because the Ph.D. process is so advisor dependent. However, doing this for each advisor is really hard too, because the sample size is small and it's hard for the reviewer to remain anonymous. All of that said, some set of general statistics (size of program, distribution of research group sizes, percentage of those who wanted a Ph.D. and then got one, etc.) plus a general satisfaction survey among graduate students might be a good complement to the usual ratings one sees in US News.
- refurb 6y ago
  Underrated comment. The “school” rating is meaningless. Even the “department” is meaningless. It all comes down to your advisor. Our department had amazing advisors and horrible advisors. Two people could be in the same PhD program and have polar opposite experiences. In fact, if you want reviews it’s not hard - just ask current students about other advisors. Rumors move fast and I’ve found you’ll get an honest opinion.
aj7 6y ago
The variation among thesis advisors within a given doctoral program is sufficiently great that it should be We need a Yelp for doctoral thesis advisors. I think the author was reluctant to “go there.”
brodo 6y ago
As someone who dropped out of a PhD program: the most important metric is the drop out rate for the research group you are in. If you can get it for your thesis advisor, it's even better. There are people who see all graduate students just as cheep, highly motivated labor for boosting their academic output. I had to put my advisor on papers who he had never read and I know someone who had to "ghost write" for her professor. Academia is a status game, and giving someone else credit for your work won't get you anywhere.
physonaught 6y ago
A good mental model for graduate school is that it is a feudal system. There is essentially no oversight of professors by a larger executive branch. I mean, they can't literally murder you, but they can drive you to suicide. (Jason Altom) If you get lucky, you get a professor that believes in you (this can be one of the most rewarding experiences I know of). If you aren't lucky, then things will get really rough. I've seen professors: - Spike their student's candidicy at the last minute (meaning the student had to switch groups in their last year, essentially having to redo their PhD in another group) - Spike student papers - finding out where their former student was applying for jobs, calling and spiking their job application (talking shit) - various abuses (yelling at student for not working on Christmas, humiliating student for years in public, sexual harassment) I could go on. If you haven't been through grad school, you might ask "why not leave/vote with your feet?" But understand the power dynamic: in really broad strokes, the student works towards a reward (being a Dr/PhD) that will take, on average, 5-6 years. Getting that reward is completely up to your supervisor's discretion. It's like if you worked at a startup for six years for minimum wage, but if your boss agrees to it, you will get a million dollar payout (also, your boss can't be fired, ever.) And that's just a minimum-- success in academia is dependent on getting full throated letters of rec from your supervisor, so they also have to really like you. So your professional success is completely up to one person: if they decide to fuck you over, that can be 4 years of work down the drain. When I was in graduate school, I never saw the faculty/department intervene in any of the abusive situations. The only thing that protects students are soft processes, like knowing which professors are really bad. The Yelp idea is good, simply because it increases that ability. And those abuses I talked about? A majority of my cohort dealt with that (at a large, well known University), it's not isolated to a few bad apples.
euix 6y ago
A PhD is a monastic experience, I remember reading that on a blog somewhere. Couldn't agree more. It comes with all the same features (it's even baked into the history of education being baked in to the Church).
croissants 6y ago
If you're debating whether to go to school x and work with professor y, make sure you get a chance to talk to students who work with professor y, and ideally students who work adjacent to professor y. In my experience, bad advisors usually have students who will tell you that the advisor is bad (especially if you talk to them in person or over the phone, where there's no written record). If the students themselves are too scared, their friends probably aren't. Any good program will have a built in process, for example an admitted students event, with this kind of interaction. Any good advisor will encourage you to talk to their students. If not, that is already a red flag. Talk to current and former students! I was fortunate to have a great advisor, and I was always happy to talk to admitted students.
devalgo 6y ago
Just want to point out that many of the complaints in the comments here are US specific. At many PhD programs in Europe for example you are considered an employee of the university and not a student, this often comes with full benefits: salary, bonuses, vacation time, pension, etc. Academia in the US in particular is falling into disrepair.
etattva 6y ago
Forget about yelp and ranking and rating. Ithink the folks realize there is not a single US website that lists all the graduate schools and the courses details. We are talking about solving the next step of the problem when basic aggregation is missing.
trombonechamp 6y ago
There was a news article about this in Science two years ago, which a couple such efforts: https://www.sciencemag.org/careers/2018/02/crowdsourcing-goes-academic-platforms-reviewing-advisers https://www.sciencemag.org/careers/2018/02/crowdsourcing-goe...
unemphysbro 6y ago
ejmr exists

180 points by etattva 6y ago

How are the data needs for such a doctoral and post-doctoral evaluation program different from the data needs for https://collegescorecard.ed.gov https://collegescorecard.ed.gov ? Data: https://collegescorecard.ed.gov/data/ https://collegescorecard.ed.gov/data/ Data documentation: https://collegescorecard.ed.gov/data/documentation/ https://collegescorecard.ed.gov/data/documentation/

sgillen 6y ago
Reviews seem like the biggest need. “The culture in the X department is terrible, everyone works 80 hours a week and is miserable” “Night life is great at least, the grad students usually go out for student nights at blah blah after seminar” “Underrated X program because at university Y, don’t be fooled we literally have 4 of the 5 top researchers in field Z” Etc.
- etattva 6y ago
  But it is so hard for people to give reviews. People seem to give good review when are they are very pissed off or very happy. Later seems to be a very rare case.
  - p1esk 6y ago
    I couldn’t make sense of your comment.
    - liammonahan 6y ago
      I think they mean “good” in the sense of being helpful reviews: positive or negative.
  - perl4ever 6y ago
    I have always thought that bad reviews are sufficient for all purposes, and the only reason to even have good reviews is so the reviewees don't feel overly persecuted. Any product or service will have occasional customers who are either unreasonable or have bad luck. If you have a lot of bad reviews, then as a prospective customer you want to see if all of them fall under the two categories or whether there is a pattern that is relevant to you. It seems to me that a review database works fine even when all the reviews are bad, and the only regulation it really needs is an effort to prevent one single individual with a vendetta from overwhelming it.

etattva 6y ago

collegescorecard measures the outcome. In the post, Profesor Leonard talks about doctoral programs being rated based on research outcomes.

michaelmrose 6y ago

So someone to extort colleges to remove bad reviews?

etattva 6y ago
Ha ha I know where you are going :) My take from the post was that there needs to be some information provided to end users based on which they should be able to decide what colleges to choose. Ranking thatis dictated by third parties cannot be a criteria. And student should not spend too much time emalining each and every college they are interested in.
- deleted 6y ago
  [deleted]

eli_gottlieb 6y ago

No. We don't need a consumer review website, because PhD students (I am one) are not the consumers of doctoral programs. We're the workers. We need a union. If we're in a good program, we're receiving valuable training both in our coursework (or before PhD in an MS/MA if your country does it that way) and in our research work. Ideally, we're building up a network of collaborations and acquaintances in the scientific community that gives us an "in" for the next stage of our career, while also being mentored by world-class experts in our fields. In a not so good program, we're used as TAs for a few years and then rapidly encouraged to drop out. Either way, most of what we do is not like most of what an undergraduate student does. It is like what an entry-level professional does.

mxcrossb 6y ago
Yeah what we really need is a glassdoor equivalent.
musicale 6y ago
> PhD students (I am one) are not the consumers of doctoral programs. We're the workers I wish I had the reference, but I remember reading about a study which showed (unsurprisingly) that the high research productivity per dollar in the US is primarily due to the below-market salaries of graduate research assistants and postdocs. Not so much capitalism - more like a guild system with a lengthy (and perhaps semi-permanent for many postdocs and adjuncts) period of apprenticeship and indenture.
chrisseaton 6y ago
On a PhD you work for yourself - you shouldn’t need a union against yourself!
- kayaeb 6y ago
  Really depends on your mentor. I've seen some gross abuses where the union was absolutely essential. Also the union hooked up like BBQs and whatever so like, that's nice.
  - 6y ago

aspaceman 6y ago

This article really rubs me the wrong way. I don't think I would find a Yelp very valuable, and the most important metric for a PhD program isn't job placement numbers. The idea that job placement correlates to PhD quality is just plain wrong. Incredibly intelligent students leave PhD programs through no fault of their own, and people who get nothing out of a PhD program can get a cushy job easily. It's a horrible metric. I found that the difference between faculty members at the same institution can be immense. This is because faculty have great control over your life due to how doctoral funding works in the United States. If you wish to continue a Biology degree, you must be willing to work with this faculty member, in this area, on this specific problem. Some faculty are rigid about a specific problem, some are not. Even more complicated is that some students need more guidance than others. Or some students want much less. And the student doesn't necessarily know what they need before entering the program. This is why I think it's smarter to give doctoral students more freedom and flexibility in switching faculty. The school should be supportive of a student in this process rather than focusing on their job placement numbers.

kyboren 6y ago
> I found that the difference between faculty members at the same institution can be immense. Absolutely! > This is why I think it's smarter to give doctoral students more freedom and flexibility in switching faculty. The school should be supportive of a student in this process. This is important, but it's not enough. The nature of a PhD means that switching faculty is often not an option. Students 3 years into their thesis work with a budding publication record on a topic often can't realistically just switch to another professor, even if everybody is on board with it. In most cases that means starting over from scratch with a new topic. So instead students will grind out a final couple (or more) miserable years in order to finally get their degree.
foobarian 6y ago
I agree with you, and further think it would be awesome if each doctoral program offered "user manuals" for each potential advisor in the department. Things like, "Professor X is very hands off, so you'll be able to do a lot on your own, but make sure you get on his calendar to get advice." Or, "Professor Y is a slave-driving task master but very capable; so make sure to stand up for yourself or you'll get crushed." Of course this is tongue in cheek and would probably never get formalized. I got this kind of advice from people who already finished their PhDs while I spent a year in industry before grad school. Not everyone can do this but maybe there is some other way to cold-contact finished PhDs and see what they know about the faculty you're considering.

deleted 6y ago

[deleted]

deleted 6y ago

[deleted]

andrewacove 6y ago

Even if you have a terrible experience in your doctoral program, much of its outward value is in the program's reputation. Giving a program a negative review hurts the perceived value of your own education.

r-w 6y ago

I think Glassdoor[1] would be a better analogue. [1] https://www.glassdoor.com/about-us/ https://www.glassdoor.com/about-us/

nabaraz 6y ago

A decent review site would be awesome. I have been looking at WGU for an online MBA degree. Reviews are just scattered everywhere from Reddit to affiliates websites.

bruceb 6y ago
Can I ask why WGU? Instead of at least a mid-tier school ?
- OldHand2018 6y ago
  WGU is accredited. If you are sufficiently motivated, it is fast and cheap. It doesn't require you to move or to quit your existing job. I can't speak to the motivations of the person considering it, but perhaps s/he only needs to check a "do you have an MBA?" box on an application to be able to move forward in their career.
- nabaraz 6y ago
  WGU is a competency based school. So I can finish my MBA from anywhere between 6 months - 3 years. On brick-mortar schools, you are limited to 3-4 classes per semester.

ISL 6y ago

Good luck getting honest reviews. Graduates and recent post-docs are dependent upon good letters of recommendation from their advisors for perhaps a decade after leaving their research groups. The truly disillusioned won't mind complaining, but those with even constructive criticism will feel constrained in their ability to speak out and be specific. The graduate-student experience is deeply advisor-specific. Professor A may be exploitative, while Professor B may fight tooth-and-nail for student success. The statistics are low, too. Most professors will only graduate a few students per decade.

musicale 6y ago
This is a good point. Anonymous reviews are not possible with such a small and easily identifiable set of reviewers.
dopu 6y ago
Completely agree. I've become increasingly convinced that we'll need a fundamental restructuring of the academy if we want to fix any of these issues. The fact that places like MIT or Stanford will hire professors simply because they output Nature papers every year, regardless of the fact that they are totally abusive to their trainees, is a perfect demonstration of it.
- lumost 6y ago
  Doesn't this explicitly favor the high-output at any cost professor? A professor who had a high nature output could attract more PhD students and structure their program such that they take more of the credit than you would expect. This wouldn't work in all fields, but there are quite a few disciplines where the biggest barrier to research is having funding and someone to do the work.
jseliger 6y ago
In many disciplines, too, the answer is "don't go to any program." Most humanities fields, for example, have numerous essays written about why doing grad school in them is a poor life choice, like my contribution: https://jakeseliger.com/2012/05/22/what-you-should-know-before-you-start-grad-school-in-english-literature-the-economic-financial-and-opportunity-costs/ https://jakeseliger.com/2012/05/22/what-you-should-know-befo.... One could say the same of most social sciences.

ramraj07 6y ago

We dont need a Yelp for doctoral programs, we need a Yelp for research labs! It's more about the lab you join than the program itself that makes or breaks your life. In fact, I have heard that minimally there's kind of a toxic labs blacklist maintained in some Chinese message boards. At least that's what a fellow postdoc told me when I mused why we suddenly stopped getting applicants from China to our lab!

frandroid 6y ago
There are PhD programs not attached to research labs. I would say the majority of doctoral students are not attached to one.
- mmhsieh 6y ago
  by research lab, he means a research group led by a specific faculty member in an academic department. in other words: he is talking about reviews on specific professors as doctoral degree advisers.
chrisseaton 6y ago
More that the lab you’re in it’s about your specific advisor. Two advisors in the same lab could work in a completely different way.
- joshvm 6y ago
  If you have an issue, the onus is on the institution to fix the problem. It doesn't matter if you're part of a lab or not, your advisor has a boss, and your university should have a support network. If it doesn't, then strongly consider going somewhere else. It also totally depends on you. I understand that in the past a lot of people clashed with my PhD supervisor, but I never had any issues as a student (and we got on well). What you need to ask is: does the instutition side with the academic in cases of misconduct (e.g. inappropriate behaviour?), or do they side with the student? Do they have a robust pastoral program? Would the students say that they feel supported? I can categorically say there are awful advisors in all institutions. A friend got a position in Cambridge. While their advisor was on vacation my friend - innocently - set up a group Slack. When the advisor returned, friend was accused of trying to steal the group. It was a bizarre situation. I know people at less well-known universities with more mundane issues, like not being supervised, and it's taken them years to get the department to do anything.

glial 6y ago

It would be better to just contact current students at the lab and offer them coffee for a conversation.

CarbyAu 6y ago
While I agree with the sentiment, many potential students are looking overseas. Is there an Internet equivalent of "coffee and conversation"? Steam game gift and discord maybe...
- henrikeh 6y ago
  You don’t have to bribe them... you can just write to them.
  - CarbyAu 6y ago
    A lot of people will say things they won't write, especially in a nice environment.
    - henrikeh 6y ago
      The schedule a call when you found someone you can talk to. Of course you lose something when physical presence is impossible, but in my experience you can always find someone who is happy to write, chat or call.

crb002 6y ago

The last time I tried to interact with Iowa State University ECprE, a professor who I won't name asked me for a $20,000 bribe to do research with their lab. Confirmed their culture is just as toxic as it was a decade ago.

ausbah 6y ago
why not nane and shame?

abakker 6y ago

We barely need a Yelp for restaurants. I think it is time for the internet to acknowledge that the pointless drivel in star rating systems is just not helping anyone. If you’re going to start a system for rating things, give it more nuance and value.

sn41 6y ago
Yup. Recently bought an affordable mechanical keyboard without consulting any review websites. Am very happy with it for about a month. Then I went to Amazon to see the reviews, out of curiosity. They are very negative for the product I am very satisfied with. Had I done due diligence, I would have lost a bunch of cash for a mere marginal improvement which none but connoisseurs can discern. I think there are a few things in life we must decide carefully - like a partner, which doctor to consult etc. Reading reviews for most of the other stuff is overthinking, and pointless.
fastball 6y ago
I actually have a plan for this! I want to build a ratings website where you rate things in the same way they generally structure psychological tests, by responding to affirmative statements with something between "strongly agree" and "strongly disagree". As an example, if you were rating films, questions could be: "This film was enjoyable." "This film had an engaging plot." "The acting was believable / appropriate for the film." etc. Ideally you'd have about 5 standard statements that you respond to for a particular thing you want to rate (whether its films, books, food, or a university course). Better than a star rating system, because the problem with stars is that you end up with scores that always hover from 5-8 (if out of 10) in a very not useful way (see IMDB). It's also better than what RT / Netflix / many others have moved to (just voting with up/down) as those require a lot of data in order to really give interesting results, where as with this system you can start to find interesting insights almost immediately, or within a single person's own collection.
- jkinudsjknds 6y ago
  People not answering questions correctly is precisely the reason many companies switch to thumbs up thumbs down. The precision of the measurement just isn't there for good / bad. Collaborative filtering with just these simple metrics works decently well, the issue is most sites like netflix have difficult UIs to actually use it as a poweruser. The average consensus of people on how good a thing is on some sort of scale, is pretty worthless. The consensus of people who seem to be like you on previous works is kind of useful. Strongly disagree, disagree, neutral, agree, strongly agree is just a 5 star system with a different name.

aleister_777 6y ago

This is hilarious. These people have contributed nothing to society, else they wouldn't required a medium to push their ideas. Your PHD means literally nothing of value!!

dhosek 6y ago

Basic rule: if you're paying for a doctorate (aside from a professional degree, like an MD or JD), it's a bad idea.

dhosek 6y ago

I've done my best to create a ranking of graduate creative writing programs by measuring the appearance of programs' graduates in the top prize anthologies. It's been generally well received although there has been some resistance to the idea of ranking programs like this. There's still a lot of work to be done on it, but the basic structure, at least is in place. http://creativewritingmfa.info/rankings/ http://creativewritingmfa.info/rankings/

etattva 6y ago
nice work. How do you keep it udpated?
- dhosek 6y ago
  There's a fair amount of manual data collection. I get an excuse to enter into e-mail correspondence with 100s of writers each year, although I have most of the initial contact programmatically driven. There's also automatic annual e-mails of program directors in August to get the latest data from them and for the top schools on the list, I'll make calls if they don't answer the e-mails. In the fall I end up spending an hour or so a day dealing with all the e-mails and data entry.

blueboo 6y ago

The problem isn’t that we’ve locked opinion inside communities. It’s that doctoral programs are so poorly run. That universities are waiting for quantitative data to instill action is more sign of dysfunction. Here’s yelp for grad programs. Write down the name of the program you’re looking at and make an asterisk. One star — it’s already more accurate than Yelp on restaurants.

vnorilo 6y ago

I am not sure a Yelp-like system would work for the reasons others have mentioned. I would suggest these metrics to help you evaluate a doc program from the outside: Do early-stage grad students get (any) authorship credits together with professors or post docs? (Getting started) Do they eventually get first author status? (Independence) Do students go to conferences and read papers? Or does a prof just go and read all 10 from her institution? (Networking) Do fresh doctors land post docs, tenure, or good private sector jobs? (Outcome) A lot of this can be deduced by google (scholar) and scanning conference proceedings and video recordings. (defended my doc in 2016 in Finland, been fairly succesful at post doc funding as well as private sector)

kayaeb 6y ago
protip: if your prof is tenured and still first author on more than 2 papers a year, they're siphoning their student's work, or they're setting you up for pipelining positions (which are not horrible, but, be aware that's what's up). I've never had a prof insert my work into their own papers, they've hooked me up to contribute to other teams, but with us it was always "my" project. If you get a chance to see them present at a conference (lots of these are on video online nowadays), check if they specifically mention their students in the presentation, that's a green flag.
- vnorilo 6y ago
  Good observation. Writing as second author was a very positive factor very early on. I was an assistant to a research fellow from my second year at uni, mostly because of coding skills. Pretty soon they encouraged me to write a pragraph or two about the implementation details, and I got added to the author list. I felt that was appropriate and a great kickstart to a research career (got my first MIT press journal credit before my bachelor) I agree however that professors growth-hacking their publication lists with student labor is problematic indeed and looking at first/second author creds is a good way to flag such behavior.

bosswipe 6y ago

There's a simple objective metric that could capture most of what you'd want to know about any school and put USNews's rankings out of business: graduates' achievement. How much are graduates making 10 years later? How many have prestigious jobs or other accomplishments such as research grants or awards? Universities do track these kinds of metrics as part of their alumni fundraising efforts but they're rarely publicly disclosed.

michaelt 6y ago
> most of what you'd want to know about any school [...] How much are graduates making 10 years later? Among my friends in graduate school: * One was told by their adviser they had enough for a thesis in their second year, and found time to organise a university-wide Dance Dance Revolution championship. * One was shouted at by their adviser for not coming in to tend to their experiments on Christmas day - the first day they'd taken off that year. I think a prospective student would like to know which of those experiences they're in store for, regardless of future earnings. Indeed, if you want to maximise your salary, I'm not sure you should go to graduate school at all...

tomashertus 6y ago

I was thinking the other day that I need a Yelp-like experience for technical tutorials and creators of educational content online. I'm sure I'm not the one who ran into so many shitty tutorials/how-to guides/videos through the years.

supernova87a 6y ago

The problem here is that a Yelp rating system for restaurants can be consumed by anyone because everyone wants a good-tasting, cheap, pleasantly-served meal. There's not a lot of range of desire and recipient goals. Grad students on the other hand, come with a huge range of: -- Motivations for doing a doctorate (say, purely for job prep, versus wanting a life of academia and intellectual fulfillment), -- Skills and knowledge (some already essentially begun their doctoral work and have achievements already, versus those who will be wandering around searching for a topic for years), -- Tolerance or willingness to engage in ambiguous, possibly dead-end leading places with no clear sign of success, -- Need for financial reward / subsistence and productive return for time or opportunity cost traded off. Add to that in many fields, the supervising professor is the greatest variable, more than the program or university -- and any rating system will be irrelevant or worse, misguiding. One person's paradise could be another's hell. Unless the rating is so dumbed down to generic factors in which case what are you really informing people about? To echo some advice of a contrarian grad school dean, go into grad school because you know what you want to write about, who you want to work with, and what you want to come out the other end with. Otherwise, the bulk majority of people going into grad school "on autopilot" or based on a Yelp rating are going to be sorely disappointed at some point.

electro_blah 6y ago

LOL You clear have no idea how Yelp works. Yelp employees and their friend will leave you bad reviews and ask you to pay (extort) in order to remove/hide them. Louis Rossmann (Search YouTube) has proof.

catsarebetter 6y ago

Doctoral programs are at least 4-5 years long and crazy emotionally challenging as well as a pretty surprising drop out rate, if someone doesn't take time to research them deeply before they commit to something like that then I think that person needs to learn the lesson that that will teach them. Yelp is the surface level easy way. Networking and cold-calling tons of current phd and master's students over the course of undergrad and during internships is how everyone does it.

ganfortran 6y ago

For real. But the title itself is something worth a moment of reflection. Doctoral programs are supposed to be scarce, selective and of high profile. Now they are being treated as a service that can be requested and compared against.

kayaeb 6y ago
This really rubbed me the wrong way, the closing sentence sums it up: "What if a doctoral program’s prestige arose, in part, from the way that it treated its students? We should dare to dream of such a thing." There is a huge problem in academia where the number of PhDs is growing exponentially, and the number of positions is staying basically flat. I strongly disagree with this "participation award" mentality of "you deserve it." It's... leading a lot of people to make really stupid career decisions and I think a lot of people are doing it just to get this gilded club to bludgeon people who don't have PhDs -- that's the only reason I can think of for a person to hope for an "easy" PhD experience. My advisor being rough at the right times (and in good spirit) was absolutely essential to the defense, which is essentially gladiatorial combat against your father (well, mother in my case), when both you and your advisor hope you can defeat the master in your specific realm. I would not have been able to do it if my advisor pranced about making everything pleasant and easy and convenient.

noelwelsh 6y ago

Any time education is discussed on HN it should come with a mandatory notice: - Your experience is no necessarily indicative of others' experiences - The US education system is not indicative of education systems in the rest of the world.

hymnsfm 6y ago

Not a fan of Yelp, but I see a need for a Yelp-like lookup for landlords and superintendents.

Upvoter33 6y ago

The problem in doing this is that saying "school X is great for getting a Ph.D." will never likely generally be true, because the Ph.D. process is so advisor dependent. However, doing this for each advisor is really hard too, because the sample size is small and it's hard for the reviewer to remain anonymous. All of that said, some set of general statistics (size of program, distribution of research group sizes, percentage of those who wanted a Ph.D. and then got one, etc.) plus a general satisfaction survey among graduate students might be a good complement to the usual ratings one sees in US News.

refurb 6y ago
Underrated comment. The “school” rating is meaningless. Even the “department” is meaningless. It all comes down to your advisor. Our department had amazing advisors and horrible advisors. Two people could be in the same PhD program and have polar opposite experiences. In fact, if you want reviews it’s not hard - just ask current students about other advisors. Rumors move fast and I’ve found you’ll get an honest opinion.

aj7 6y ago

The variation among thesis advisors within a given doctoral program is sufficiently great that it should be We need a Yelp for doctoral thesis advisors. I think the author was reluctant to “go there.”

brodo 6y ago

As someone who dropped out of a PhD program: the most important metric is the drop out rate for the research group you are in. If you can get it for your thesis advisor, it's even better. There are people who see all graduate students just as cheep, highly motivated labor for boosting their academic output. I had to put my advisor on papers who he had never read and I know someone who had to "ghost write" for her professor. Academia is a status game, and giving someone else credit for your work won't get you anywhere.

physonaught 6y ago

A good mental model for graduate school is that it is a feudal system. There is essentially no oversight of professors by a larger executive branch. I mean, they can't literally murder you, but they can drive you to suicide. (Jason Altom) If you get lucky, you get a professor that believes in you (this can be one of the most rewarding experiences I know of). If you aren't lucky, then things will get really rough. I've seen professors: - Spike their student's candidicy at the last minute (meaning the student had to switch groups in their last year, essentially having to redo their PhD in another group) - Spike student papers - finding out where their former student was applying for jobs, calling and spiking their job application (talking shit) - various abuses (yelling at student for not working on Christmas, humiliating student for years in public, sexual harassment) I could go on. If you haven't been through grad school, you might ask "why not leave/vote with your feet?" But understand the power dynamic: in really broad strokes, the student works towards a reward (being a Dr/PhD) that will take, on average, 5-6 years. Getting that reward is completely up to your supervisor's discretion. It's like if you worked at a startup for six years for minimum wage, but if your boss agrees to it, you will get a million dollar payout (also, your boss can't be fired, ever.) And that's just a minimum-- success in academia is dependent on getting full throated letters of rec from your supervisor, so they also have to really like you. So your professional success is completely up to one person: if they decide to fuck you over, that can be 4 years of work down the drain. When I was in graduate school, I never saw the faculty/department intervene in any of the abusive situations. The only thing that protects students are soft processes, like knowing which professors are really bad. The Yelp idea is good, simply because it increases that ability. And those abuses I talked about? A majority of my cohort dealt with that (at a large, well known University), it's not isolated to a few bad apples.

euix 6y ago

A PhD is a monastic experience, I remember reading that on a blog somewhere. Couldn't agree more. It comes with all the same features (it's even baked into the history of education being baked in to the Church).

croissants 6y ago

If you're debating whether to go to school x and work with professor y, make sure you get a chance to talk to students who work with professor y, and ideally students who work adjacent to professor y. In my experience, bad advisors usually have students who will tell you that the advisor is bad (especially if you talk to them in person or over the phone, where there's no written record). If the students themselves are too scared, their friends probably aren't. Any good program will have a built in process, for example an admitted students event, with this kind of interaction. Any good advisor will encourage you to talk to their students. If not, that is already a red flag. Talk to current and former students! I was fortunate to have a great advisor, and I was always happy to talk to admitted students.

devalgo 6y ago

Just want to point out that many of the complaints in the comments here are US specific. At many PhD programs in Europe for example you are considered an employee of the university and not a student, this often comes with full benefits: salary, bonuses, vacation time, pension, etc. Academia in the US in particular is falling into disrepair.

etattva 6y ago

Forget about yelp and ranking and rating. Ithink the folks realize there is not a single US website that lists all the graduate schools and the courses details. We are talking about solving the next step of the problem when basic aggregation is missing.

trombonechamp 6y ago

There was a news article about this in Science two years ago, which a couple such efforts: https://www.sciencemag.org/careers/2018/02/crowdsourcing-goes-academic-platforms-reviewing-advisers https://www.sciencemag.org/careers/2018/02/crowdsourcing-goe...

unemphysbro 6y ago

ejmr exists