6 ms·

Using machine learning to predict the leads that close

57 points by standrews 4y ago

tdeck 4y ago
Why does the X-axis on the marketing email count graph to up to 60? Is this where we've ended up, sending 60 emails to the same lead? Ouch.
- standrews 4y ago
  It's someone opening or clicking emails 60 times (can happen!), not sending 60 marketing emails.
  - tdeck 4y ago
    With Gmail and the like you only get to know about the first open. I wonder if this is actually caused by the email being forwarded around to many people.
    - standrews 4y ago
      Email forwarding or just quirks of various email services and clients. Email clicks or website visits definitely a more reliable signal
standrews 4y ago
Which sales leads to focus on? We were in a unique position to answer this because we connect data from marketing tools like MailChimp with CRMs like Pipedrive, plus we track web visits.
CarbonCycles 4y ago
I like your blog on how you use EDA, but I'm not sure I'm getting the Machine Learning piece. It would be nice if you guys went into more details, but I appreciate how you were able to tie together different data sources and walked ppl through the analysis!
- malshe 4y ago
  I am guessing they used logistic regression because they could show the lead win probability as a function of emails opened. It's quite easy to do with logistic regression.
  - capn_duck 4y ago
    nah its ml bro
- standrews 4y ago
  There's a separate technical post coming - will post a link here once it's live.
shcheklein 4y ago
I worked previously at a company that was trying to do a predictive analytic, ML on top of Salesforce (to qualify and score leads, generate new promising leads, etc). The task was extremely difficult and from what I saw people were getting some marginal improvements from ML itself. Most companies at the end were interested in basic (but quite hard things to do right) - cleaning, dedupoing, enriching, generating more leads based on ICP, etc. Not sure if in this case it's something different but my take from the past experience it's hard since data is noisy + closing the deals depends on a human as well and these tools don't take that into account usually.
- shcheklein 4y ago
  I hope though there will be an ML product that can do a decent improvement in this space. Even small improvement there brings significant ROI.
  - kjellsbells 4y ago
    I spent years using a tool called Clari that applied analytics to Salesforce and Office (email) to predict deal activity. When it worked, it was really good. But it had a fatal flaw: relying on Salesforce meant relying on the data that sales people input. And I quickly learned that sales people hate reporting tools, update them only under duress, and generally fill the database with crap. Perhaps it's different when you are selling subscriptions over a digital channel, but for classic B2B feet on the street deals, ugh.
    - asyncscrum 4y ago
      Thus spawning a whole new class of tools that add a more convenient interface on-top of salesforce.
      - kjellsbells 4y ago
        Right, but garbage in, garbage out, whether you are feeding your CRM directly or via a shiny UI layer. No amount of ML can save you if the sales droid isnt filing data.
unlikelymordant 4y ago
What should you do with this information though? Should the salespeople focus on the customers that are most likely to convert? Or should the salespeople give minimal attention to those, and focus most on the ones less likely, but not 0 probability, to convert. I think the most important outputs of this are understanding the factors involved in conversion to tune business processes, not necessarily using the outputs of the model to target specific users.
- rukuu001 4y ago
  Yeah, the salespeople avoid spending time on leads unlikely to close. They're all operating on this kind of information already (like the basic stuff - is this person a decision-maker, do they have budget etc etc). I take this seriously because I've seen consultants making money in this space already (advising on leads unlikely to close).
  - roenxi 4y ago
    I think I agree, and we could make exactly that observation about syntax highlighting. It isn't like syntax highlighting is telling me things I already know, but it is a free win for my productivity while coding. Making something that is already obvious cheaper to discover can help the people who need to act on it.
  - 6stringmerc 4y ago
    Maybe it’s like the nuance in music - knowing the notes not to play. I think the art is in weighing the variables as a human because they can change also based on competition in the room. Having a career ongoing in sales support I’ve seen first hand how erratic the decision trees can be for private or public organizations. While the general setup is similar “do business with X with Y” the ingredients can differ widely to get to the sale. ML might tell you to send a holiday gift but I bet the human has a better idea of what kind of gift to send than ML.
  - jross225 4y ago
    But then this has the impact of reenforcing model predictions (unless controlled for). This lead won't convert based on the model, so I won't spend time with them or give them any preferential pricing, so the lead doesn't close (and the converse is true). This isn't a pure ML problem, and without "treatment" data I'm not quite sure how the blog is adjusting for customer propensity towards an outcome :/
fredley 4y ago
In Shakespeare's Macbeth, the witches prophecise to him that he will become King. Spoiler alert, he does, by killing the previous King in order to make the prophecy come true.

57 points by standrews 4y ago

Why does the X-axis on the marketing email count graph to up to 60? Is this where we've ended up, sending 60 emails to the same lead? Ouch.

standrews 4y ago
It's someone opening or clicking emails 60 times (can happen!), not sending 60 marketing emails.
- tdeck 4y ago
  With Gmail and the like you only get to know about the first open. I wonder if this is actually caused by the email being forwarded around to many people.
  - standrews 4y ago
    Email forwarding or just quirks of various email services and clients. Email clicks or website visits definitely a more reliable signal

standrews 4y ago

Which sales leads to focus on? We were in a unique position to answer this because we connect data from marketing tools like MailChimp with CRMs like Pipedrive, plus we track web visits.

CarbonCycles 4y ago

I like your blog on how you use EDA, but I'm not sure I'm getting the Machine Learning piece. It would be nice if you guys went into more details, but I appreciate how you were able to tie together different data sources and walked ppl through the analysis!

malshe 4y ago
I am guessing they used logistic regression because they could show the lead win probability as a function of emails opened. It's quite easy to do with logistic regression.
- capn_duck 4y ago
  nah its ml bro
standrews 4y ago
There's a separate technical post coming - will post a link here once it's live.

shcheklein 4y ago

I worked previously at a company that was trying to do a predictive analytic, ML on top of Salesforce (to qualify and score leads, generate new promising leads, etc). The task was extremely difficult and from what I saw people were getting some marginal improvements from ML itself. Most companies at the end were interested in basic (but quite hard things to do right) - cleaning, dedupoing, enriching, generating more leads based on ICP, etc. Not sure if in this case it's something different but my take from the past experience it's hard since data is noisy + closing the deals depends on a human as well and these tools don't take that into account usually.

shcheklein 4y ago
I hope though there will be an ML product that can do a decent improvement in this space. Even small improvement there brings significant ROI.
- kjellsbells 4y ago
  I spent years using a tool called Clari that applied analytics to Salesforce and Office (email) to predict deal activity. When it worked, it was really good. But it had a fatal flaw: relying on Salesforce meant relying on the data that sales people input. And I quickly learned that sales people hate reporting tools, update them only under duress, and generally fill the database with crap. Perhaps it's different when you are selling subscriptions over a digital channel, but for classic B2B feet on the street deals, ugh.
  - asyncscrum 4y ago
    Thus spawning a whole new class of tools that add a more convenient interface on-top of salesforce.
    - kjellsbells 4y ago
      Right, but garbage in, garbage out, whether you are feeding your CRM directly or via a shiny UI layer. No amount of ML can save you if the sales droid isnt filing data.

unlikelymordant 4y ago

What should you do with this information though? Should the salespeople focus on the customers that are most likely to convert? Or should the salespeople give minimal attention to those, and focus most on the ones less likely, but not 0 probability, to convert. I think the most important outputs of this are understanding the factors involved in conversion to tune business processes, not necessarily using the outputs of the model to target specific users.

rukuu001 4y ago
Yeah, the salespeople avoid spending time on leads unlikely to close. They're all operating on this kind of information already (like the basic stuff - is this person a decision-maker, do they have budget etc etc). I take this seriously because I've seen consultants making money in this space already (advising on leads unlikely to close).
- roenxi 4y ago
  I think I agree, and we could make exactly that observation about syntax highlighting. It isn't like syntax highlighting is telling me things I already know, but it is a free win for my productivity while coding. Making something that is already obvious cheaper to discover can help the people who need to act on it.
- 6stringmerc 4y ago
  Maybe it’s like the nuance in music - knowing the notes not to play. I think the art is in weighing the variables as a human because they can change also based on competition in the room. Having a career ongoing in sales support I’ve seen first hand how erratic the decision trees can be for private or public organizations. While the general setup is similar “do business with X with Y” the ingredients can differ widely to get to the sale. ML might tell you to send a holiday gift but I bet the human has a better idea of what kind of gift to send than ML.
- jross225 4y ago
  But then this has the impact of reenforcing model predictions (unless controlled for). This lead won't convert based on the model, so I won't spend time with them or give them any preferential pricing, so the lead doesn't close (and the converse is true). This isn't a pure ML problem, and without "treatment" data I'm not quite sure how the blog is adjusting for customer propensity towards an outcome :/

fredley 4y ago

In Shakespeare's Macbeth, the witches prophecise to him that he will become King. Spoiler alert, he does, by killing the previous King in order to make the prophecy come true.