8 ms·

Stack Exchange Engineering: How We Built Our Blog

81 points by jonhmchan 11y ago

danso 11y ago
Spoiler alert: it takes awhile to get there, but the answer is: Jekyll. Which is neat...even as active as their blogging efforts are, as I was reading the intro grafs, I kept thinking, "Yeah, could be done just fine as Jekyll"...I was mostly expecting them to announce new in-house built blogging software, but was thrilled to see that the rest of the post is about how to migrate from WordPress to Jekyll.
allendoerfer 11y ago
They lost a simple interface designed to edit blog posts and now their marketing team has to use the Github interface and learn Markdown. They lost the comment system and had to move the data to an external provider, which instantly bit them. They had to write Python scripts to munge the data. But now they have the performance of static html. I guess just changing two lines in the nginx configuration did not came to mind. I think this was a very poor investment and will never pay off.
- swasheck 11y ago
  i dislike wordpress quite a bit, but i also agree that this was a head-scratching investment.
- toast0 11y ago
  Last year, I took two days to build a mostly static blog to replace WordPress. It pays off everytime there is a WordPress vulnerability I don't have to worry about, and everytime we post an interesting post and I don't have to worry about if WordPress is going to fall over (yes, maybe there's some caching thing we could have used). It pays off when I have multiple data centers running, because WordPress doesn't have the ability to read from a local MySQL slave, so perf on the datacenter far from the master is terrible (the i18n plugin we were using didn't help). I imagine the payoff is similar for the stack overflow team as well. EDIT: I don't understand why they are using disqus for comments, when they have a self hosted comment platform, in my case we don't accept comments.
  - hueving 11y ago
    Simple caching would have solved all of your issues. Classic case of not profiling before blaming the whole system.
    - toast0 11y ago
      Simple caching wouldn't have solved the (numerous) security issues.
      - wink 11y ago
        You could still export/dump the WordPress pages as static html and still keep the editor. (and WordPress not accessible from the outside).
- tsurantino 11y ago
  Asana did something similar: https://eng.asana.com/2014/02/scaling-asana-com/ https://eng.asana.com/2014/02/scaling-asana-com/ There is an argument to be made that a ubiquitous, developer-first system translates into more instant changes. Seeing changes quickly makes it easier to iterate, which is better over time than the initial investment to learn Markdown.
- smoser 11y ago
  The marketing team can use http://prose.io http://prose.io without learning Markdown. Using using an external comment provider promotes some popular users to comment such as Grimlock whom otherwise wouldn't.
  - jonhmchan 11y ago
    Someone else also mentioned prose.io - something I didn't come across when building this. Going to explore.
- alexa_scordato 11y ago
  Hi! I'm the Director of Product Marketing at Stack Exchange. It might sound like more work for us to use Github and markdown, but I see it as a huge plus. For non technical employees, this is a great opportunity to learn a new skill and collaborate with the engineering organization. Jon trained a few of us already... so far, so good!
  - allendoerfer 11y ago
    Hi! While the change surely was not a huge investment I just do not believe that the benefits justify the costs. Since you wrote a blog post about it, I felt free to question your decisions. Nevertheless, now that it is done, I hope the new system works well for you. Maybe your team gets used to git for version control in other areas of work, too, which would be a huge win and ultimately prove me wrong ;-)
    - alexa_scordato 11y ago
      I hope it works well too! Maybe I'll write a follow-up blog post in a few months and report back on our progress. In general, I think that marketers today should acquire some basic front-end knowledge and that knowing git and markdown is a good step towards getting there. Coding things like email templates and landing pages without the help of a designer or developer will be huge for us.
      - mattmanser 11y ago
        You will never be able to code email templates and that just shows how clueless you are as to the rabbit hole your devs have got you in to. Email markup is so arcane and complex at the moment that one mistake by your non-technical team will mean hours of pain. I know you want to support your colleagues, but they have made a bad decision which will make your job less doing the things you are good at and more dealing with things that should never have been your problem.
        jc4p 11y ago
- fsk 11y ago
  My employer originally had a Wordpress-based blogging product. They spent more than $2M writing their own blogging system, and it still was missing many features that come with Wordpress. They ditched it and switched back to Wordpress.
  - jlangenauer 11y ago
    TWO MILLION DOLLARS on a blogging system. I'm speechless.
    - morgante 11y ago
      It might surprise you even more to know that the creators of the most popular blogging system have raise $317M.
    - hueving 11y ago
      10 silicon Valley software engineers on something goes through 2 million in under a year when you consider total cost of employment.
      - jon-wood 11y ago
        If it takes 10 engineers a year to build a blog then something is very wrong.
        rokhayakebe 11y ago
        I would like to see a study where one would give the same project with a solid deadline to three different teams of 3, 15, 30 engineers and see how long it takes each them to finish it.
- jonhmchan 11y ago
  I'm the author of the post and the lead on this project. Many of the costs you mention aren't so bad. As mentioned earlier, we actually consider learning GH and Markdown a plus for non-devs. As for comments and writing Python, some pain here, but were ultimately solvable. Performance also wasn't the only plus here. Closing major security holes, making more of our content and technology more open, and moving to a platform that our devs liked working in are just some of the other wins. It's too early to say definitively now, but we think the change is probably a good one.
  - clumsysmurf 11y ago
    Did anyone evaluate ghost? If so, I'm curious what the verdict was. https://ghost.org/ https://ghost.org/
    - jc4p 11y ago
      We did consider Ghost, I love it and it would've given us uploading images to S3 for free, but static files and being able to use any markdown editor you want won over.
  - rquirk 11y ago
    You haven't broken the podcast RSS feed, thank you. Another large podcast network recently did a redesign and broke all of their feeds, apparently on purpose. Crazy. However, the podcast feed is impossible to find now if you didn't have it before. Previously I think it was on every podcast post as an RSS link. Now the individual tag RSS links all point to the main feed, rather than to a per-tag feed. e.g. http://blog.stackoverflow.com/category/podcasts/feed/ http://blog.stackoverflow.com/category/podcasts/feed/ still works if you know it, but going to https://blog.stackexchange.com/tags/podcasts/ https://blog.stackexchange.com/tags/podcasts/ gives you an RSS link to just "/feed/". This is probably not a good place for bug reports :-S
- nnq 11y ago
  They've also lost COMPLEXITY, and this always pays off, even if in roundabout ways that you cannot measure, and even if it's only for the positive effects on the mental-health of the engineers working on this! Really... modern WordPress is a very complicated beast and keeping all sorts of details about how you need to optimize it, server, caching, db in you head is just no worth it. Yeah, "changing two lines in the nginx configuration" could have worked, but imho simply the cost of keeping in your head the information about what two lines to change is just too large. As it grows, it simply drives you insane by how many technical trivialities you need to keep in your head to keep it working smoothly, when you could instead use that brainpower so solve the really interesting problems. An the fact that "their marketing team has to use the Github interface and learn Markdown" is also a hidden benefit, if you think about all the subtle frontend bugs that happen when somebody uses the WYSIWYG editor of WP and they end up with a combination of styles that breaks the layout. For any project above a certain size, the "dumb WIM" (What I Mean) interface of Markdown always trumps the WYSIWYG. Also, the benefit of mostly NOT HAVING TO THINK ABOUT SECURITY AT ALL is freaking HUGE! Developers can simply say "it's the server guys' problem" now. Forget about having to manually audit the code of 3rd party plugins if you really care about security. And forget about updating and realizing something breaks and you have to fix it. For any fast growing company, keep this in mind: REDUCE COMPLEXITY. ALWAYS. NO MATTER HOW LARGE THE SHORT TERM COST.
  - pp19dd 11y ago
    Just as an aside to WordPress optimization, so there are two ways to do that. One of them is to analyze content flow (how often are things updated) and traffic flow and then come up with a reasonable strategy. Things like use memcaching for sidebars, separate those out depending whether you need to, then work on caching plugins. The caching plugins need work because they tend to be programmed to allow robots to spider uncached content, which makes no sense to me. Robot traffic on high volume sites is just insane. With a hundred blogs or so on a multisite installation you could easily spend a month doing postmortems and hunting down the least common denominator (oh hello switch_to_blog, why do you use 36 MB of RAM on each call?) Or, you can slap some ram sticks in the machine, install varnish-cache, write a few rules for it and go play minecraft the other 29 days.
  - 11y ago
- fabian2k 11y ago
  Stack Exchange uses Markdown for everything on their Q&A sites already. If they expect their users to be able to use it, I think they can expect the same from their employees. It's likely not a wasted effort to learn it as they might need it somewhere else as well. The Stack Exchange blog was hacked a few years ago, I suspect that was part of the motivation to switch to something they know better.
ne01 11y ago
At SunSed we are trying to solve this exact problem: A blogging platform for professional whith batteries included. Kind of excited to see WordPress does not work for everyone. Thanks for sharing!
dylz 11y ago
Damn. Was hoping for a self hosted (and full featured) disqus alternative or something to appear one day.
- jonhmchan 11y ago
  Discourse was really, really close for us. The breaker was the lack of a real WordPress import support: https://meta.discourse.org/t/is-it-possibile-to-import-existing-wordpress-comments-along-with-post-into-discourse/18157 https://meta.discourse.org/t/is-it-possibile-to-import-exist... That being said, that evaluation was done before we found out how difficult it would be with Disqus.
cddotdotslash 11y ago
I'm all for spending engineering time on non-project related tasks occasionally, but this seems like an excessive use of valuable time to reinvent the wheel. I'm sure they could have found a blogging platform that didn't require custom markdown interfaces, Python scripting, and re-writing everything from scratch. It's just a blog.
- jacques_chester 11y ago
  An old unix greybeard of my acquaintance told me circa 2004 that "writing a blog engine is this generation's version of writing a shell". Creation is more fun than selection. And in the same position, I'd ditch Wordpress like a stolen sports car.
- hayksaakian 11y ago
  I disagree. It seems like the biggest time-wasters were exporting all their data from wordpress without breaking anything. Even if they found "a blogging platform that iddn't require custom markdown interfaces" that would not have solved the problems they faced getting data out of wordpress.
  - hueving 11y ago
    So why didn't they just fix WordPress. If export was expensive and they were willing to burn engineering time on it, the solution seems obvious. However, WordPress is not cool so they went to the tired old "reinvent blogging". No open source contributions, just building castles in their own sandbox.
    - jacques_chester 11y ago
      Wordpress is filled with third rails.
    - danso 11y ago
      I don't want to make this sound like an appeal to authority...but have you created/maintained both WordPress and static-generated blogs (either Jekyll, or something similar like Middleman)? Without having done both, I'd tend to agree with you: a blog is a blog. The opportunity cost from system migration (and re-invention) is non-trivial. But so are the opportunities gained with a new infrastructure. The speed gains from serving static files is nice, but to me, they are no where near the kind of gains I make by being able to work with plain text files. When I want to write a file, I pop open my (probably already open) text editor, bang it out, save, switch to Terminal, push to S3. Did I make a typo? OK, fix the typo, save, re-run the Terminal command. Maybe I want to push a few posts, but I want to make sure they all look right...OK, run the jekyll server and look at localhost. Then run the push command. Oh wait, maybe one of those posts should be a draft. OK, set it to draft in the metadata, re-push, etc. etc. I have a particularly bad experience with WordPress because I've been running my blog on it for five years on the cheapest Dreamhost shared server. However, I've spent a good amount of time researching and installing WordPress plugins to do caching...but the problem is that every goddamnned database action, including setting a post from draft to publish, or vice versa, is a 5 to 50 second process. Obviously, Stack Exchange doesn't suffer under that kind of hosting issue. And they could probably set up their WordPress on a local instance so that it can be locally browsed (in my experience, this can be profoundly difficult, and quite the weekend project). But no matter what their hosting infrastructure is, it still doesn't negate the fact that every posting action requires a database -- everything from creating users, setting user roles, setting post status, and of course, editing the post. Sure, you have to learn those things in Jekyll too...but the conventions are easy...and most importantly, you can do them within the confines of your text editor. That means everyone who contributes to the blog, engineers and marketers, gets better at their text editor (and operating system). This is a much more worthwhile skill than getting better at WordPress...which is a non-trivial skill...WordPress, to its credit, is fast moving in its development...for occasional publishers like me, this means I spent a good amount of cognitive energy re-learning the interface. With Jekyll, even as the software changes, I just have to learn my text editor. And that's just wonderful, considering how much of my work involves a text editor. For non-techies, I imagine it's a little daunting at first...but they're fully-functioning adults. They can handle plain text. And working with Jekyll not only makes it easier for the engineers to do the occasional hacking of templates, it allows the non-techies to actually _see_ it without having to jump through the many plugin submenus in the WordPress admin. It is literally a Cmd-T (in Sublime Text) to look up a file. Or, a Find All across a project...which is another huge benefit...blog contributors often need to look up past material...the text editor search is far superior to what can be done in the WP Admin or other web search interfaces. So it's not that "WordPress is not cool"...and the fact that you think that one solution is to "just fix WordPress" makes me think you don't understand the problem. WordPress doesn't need to be "fixed" to be more like Jekyll, it is a fundamentally different product, with certain tradeoffs and benefits worth pursing. To say that the OP should've "just fixed WordPress" is like saying, "Why use Facebook? Just get better at using the CC: field in email"
jlarocco 11y ago
It seems like this was a big waste of time reinventing Octopress.
- tarr11 11y ago
  Octopress is very difficult to use, and I would not recommend it. Using jekyll directly is a much better choice.
  - jacquesm 11y ago
    Octopress is built on top of jekyll and what is a real nuisance is how long it takes to re-generate a site with more than a few posts.
    - Nick-Craver 11y ago
      This isn't true anymore. Octopress uses an older version of Jekyll. If you're on 3.0 (which we're using) has incremental generation. Here's the log of a recent incremental build: [Step 1/3] Jekyll build (11s) [20:52:40][Jekyll build] Jekyll build --config '_blog.config.yml' --no-watch [20:52:43][Jekyll build] Configuration file: C:/BuildAgent/work/4e27a39603eb2bf1/blog/_config.yml [20:52:43][Jekyll build] Source: C:/BuildAgent/work/4e27a39603eb2bf1/blog [20:52:43][Jekyll build] Destination: C:/BuildAgent/work/4e27a39603eb2bf1/blog/_site [20:52:43][Jekyll build] Incremental build: disabled [20:52:43][Jekyll build] Generating... [20:52:52][Jekyll build] done in 9.082 seconds.
      - jacquesm 11y ago
        Cool, time to upgrade then!
misterjinx 11y ago
It is not possible to subscribe to feeds individually ? For instance I would like to subscribe only for the engineering RSS feed. At the moment all I could find was a general feed for the entire blog.
vlad 11y ago
That’s exactly what I just did for my daily video / podcasting site Rate My App.com. - Everything in markdown - Generate the podcasts as an RSS feed - Generate the list of videos as JSON, filtered inside the AngularJS app Such an approach is not great for SEO in the short-term, so I won't be surprised if they lose ranking because their posts and comments are missing from the the html until their jQuery code fetches and renders them. For me, I don't have 700 posts so that's easy to address later once I have content worth indexing, keeping the urls the same of course. :)
- chaselee 11y ago
  Check out [prerender](https://prerender.io/ https://prerender.io/) to get around the SEO issue.
kmfrk 11y ago
Static blogs are doing pretty well, but setting up a sepparate interface or flow for writers - not admins and developers - is still a big of a chore, unless you don’t mind giving people full access to the site repo. I’m thinking about setting up a separate draft and post repo and importing them into the root with git subtree.
philippnagel 11y ago
Are there any good, corporate solutions for organizations that lack in-house engineering resources?
- saosebastiao 11y ago
  Jekyll is pretty low impact as far as engineering goes...all the engineering work here went into migration of the existing blog. I would think getting Jekyll set up would be even easier than a self-hosted Wordpress installation.
rebootthesystem 11y ago
They had me at "Not Wordpress or PHP"

81 points by jonhmchan 11y ago

danso 11y ago

Spoiler alert: it takes awhile to get there, but the answer is: Jekyll. Which is neat...even as active as their blogging efforts are, as I was reading the intro grafs, I kept thinking, "Yeah, could be done just fine as Jekyll"...I was mostly expecting them to announce new in-house built blogging software, but was thrilled to see that the rest of the post is about how to migrate from WordPress to Jekyll.

allendoerfer 11y ago

They lost a simple interface designed to edit blog posts and now their marketing team has to use the Github interface and learn Markdown. They lost the comment system and had to move the data to an external provider, which instantly bit them. They had to write Python scripts to munge the data. But now they have the performance of static html. I guess just changing two lines in the nginx configuration did not came to mind. I think this was a very poor investment and will never pay off.

swasheck 11y ago
i dislike wordpress quite a bit, but i also agree that this was a head-scratching investment.
toast0 11y ago
Last year, I took two days to build a mostly static blog to replace WordPress. It pays off everytime there is a WordPress vulnerability I don't have to worry about, and everytime we post an interesting post and I don't have to worry about if WordPress is going to fall over (yes, maybe there's some caching thing we could have used). It pays off when I have multiple data centers running, because WordPress doesn't have the ability to read from a local MySQL slave, so perf on the datacenter far from the master is terrible (the i18n plugin we were using didn't help). I imagine the payoff is similar for the stack overflow team as well. EDIT: I don't understand why they are using disqus for comments, when they have a self hosted comment platform, in my case we don't accept comments.
- hueving 11y ago
  Simple caching would have solved all of your issues. Classic case of not profiling before blaming the whole system.
  - toast0 11y ago
    Simple caching wouldn't have solved the (numerous) security issues.
    - wink 11y ago
      You could still export/dump the WordPress pages as static html and still keep the editor. (and WordPress not accessible from the outside).
tsurantino 11y ago
Asana did something similar: https://eng.asana.com/2014/02/scaling-asana-com/ https://eng.asana.com/2014/02/scaling-asana-com/ There is an argument to be made that a ubiquitous, developer-first system translates into more instant changes. Seeing changes quickly makes it easier to iterate, which is better over time than the initial investment to learn Markdown.
smoser 11y ago
The marketing team can use http://prose.io http://prose.io without learning Markdown. Using using an external comment provider promotes some popular users to comment such as Grimlock whom otherwise wouldn't.
- jonhmchan 11y ago
  Someone else also mentioned prose.io - something I didn't come across when building this. Going to explore.
alexa_scordato 11y ago
Hi! I'm the Director of Product Marketing at Stack Exchange. It might sound like more work for us to use Github and markdown, but I see it as a huge plus. For non technical employees, this is a great opportunity to learn a new skill and collaborate with the engineering organization. Jon trained a few of us already... so far, so good!
- allendoerfer 11y ago
  Hi! While the change surely was not a huge investment I just do not believe that the benefits justify the costs. Since you wrote a blog post about it, I felt free to question your decisions. Nevertheless, now that it is done, I hope the new system works well for you. Maybe your team gets used to git for version control in other areas of work, too, which would be a huge win and ultimately prove me wrong ;-)
  - alexa_scordato 11y ago
    I hope it works well too! Maybe I'll write a follow-up blog post in a few months and report back on our progress. In general, I think that marketers today should acquire some basic front-end knowledge and that knowing git and markdown is a good step towards getting there. Coding things like email templates and landing pages without the help of a designer or developer will be huge for us.
    - mattmanser 11y ago
      You will never be able to code email templates and that just shows how clueless you are as to the rabbit hole your devs have got you in to. Email markup is so arcane and complex at the moment that one mistake by your non-technical team will mean hours of pain. I know you want to support your colleagues, but they have made a bad decision which will make your job less doing the things you are good at and more dealing with things that should never have been your problem.
      - jc4p 11y ago
fsk 11y ago
My employer originally had a Wordpress-based blogging product. They spent more than $2M writing their own blogging system, and it still was missing many features that come with Wordpress. They ditched it and switched back to Wordpress.
- jlangenauer 11y ago
  TWO MILLION DOLLARS on a blogging system. I'm speechless.
  - morgante 11y ago
    It might surprise you even more to know that the creators of the most popular blogging system have raise $317M.
  - hueving 11y ago
    10 silicon Valley software engineers on something goes through 2 million in under a year when you consider total cost of employment.
    - jon-wood 11y ago
      If it takes 10 engineers a year to build a blog then something is very wrong.
      - rokhayakebe 11y ago
        I would like to see a study where one would give the same project with a solid deadline to three different teams of 3, 15, 30 engineers and see how long it takes each them to finish it.
jonhmchan 11y ago
I'm the author of the post and the lead on this project. Many of the costs you mention aren't so bad. As mentioned earlier, we actually consider learning GH and Markdown a plus for non-devs. As for comments and writing Python, some pain here, but were ultimately solvable. Performance also wasn't the only plus here. Closing major security holes, making more of our content and technology more open, and moving to a platform that our devs liked working in are just some of the other wins. It's too early to say definitively now, but we think the change is probably a good one.
- clumsysmurf 11y ago
  Did anyone evaluate ghost? If so, I'm curious what the verdict was. https://ghost.org/ https://ghost.org/
  - jc4p 11y ago
    We did consider Ghost, I love it and it would've given us uploading images to S3 for free, but static files and being able to use any markdown editor you want won over.
- rquirk 11y ago
  You haven't broken the podcast RSS feed, thank you. Another large podcast network recently did a redesign and broke all of their feeds, apparently on purpose. Crazy. However, the podcast feed is impossible to find now if you didn't have it before. Previously I think it was on every podcast post as an RSS link. Now the individual tag RSS links all point to the main feed, rather than to a per-tag feed. e.g. http://blog.stackoverflow.com/category/podcasts/feed/ http://blog.stackoverflow.com/category/podcasts/feed/ still works if you know it, but going to https://blog.stackexchange.com/tags/podcasts/ https://blog.stackexchange.com/tags/podcasts/ gives you an RSS link to just "/feed/". This is probably not a good place for bug reports :-S
nnq 11y ago
They've also lost COMPLEXITY, and this always pays off, even if in roundabout ways that you cannot measure, and even if it's only for the positive effects on the mental-health of the engineers working on this! Really... modern WordPress is a very complicated beast and keeping all sorts of details about how you need to optimize it, server, caching, db in you head is just no worth it. Yeah, "changing two lines in the nginx configuration" could have worked, but imho simply the cost of keeping in your head the information about what two lines to change is just too large. As it grows, it simply drives you insane by how many technical trivialities you need to keep in your head to keep it working smoothly, when you could instead use that brainpower so solve the really interesting problems. An the fact that "their marketing team has to use the Github interface and learn Markdown" is also a hidden benefit, if you think about all the subtle frontend bugs that happen when somebody uses the WYSIWYG editor of WP and they end up with a combination of styles that breaks the layout. For any project above a certain size, the "dumb WIM" (What I Mean) interface of Markdown always trumps the WYSIWYG. Also, the benefit of mostly NOT HAVING TO THINK ABOUT SECURITY AT ALL is freaking HUGE! Developers can simply say "it's the server guys' problem" now. Forget about having to manually audit the code of 3rd party plugins if you really care about security. And forget about updating and realizing something breaks and you have to fix it. For any fast growing company, keep this in mind: REDUCE COMPLEXITY. ALWAYS. NO MATTER HOW LARGE THE SHORT TERM COST.
- pp19dd 11y ago
  Just as an aside to WordPress optimization, so there are two ways to do that. One of them is to analyze content flow (how often are things updated) and traffic flow and then come up with a reasonable strategy. Things like use memcaching for sidebars, separate those out depending whether you need to, then work on caching plugins. The caching plugins need work because they tend to be programmed to allow robots to spider uncached content, which makes no sense to me. Robot traffic on high volume sites is just insane. With a hundred blogs or so on a multisite installation you could easily spend a month doing postmortems and hunting down the least common denominator (oh hello switch_to_blog, why do you use 36 MB of RAM on each call?) Or, you can slap some ram sticks in the machine, install varnish-cache, write a few rules for it and go play minecraft the other 29 days.
- 11y ago
fabian2k 11y ago
Stack Exchange uses Markdown for everything on their Q&A sites already. If they expect their users to be able to use it, I think they can expect the same from their employees. It's likely not a wasted effort to learn it as they might need it somewhere else as well. The Stack Exchange blog was hacked a few years ago, I suspect that was part of the motivation to switch to something they know better.

ne01 11y ago

At SunSed we are trying to solve this exact problem: A blogging platform for professional whith batteries included. Kind of excited to see WordPress does not work for everyone. Thanks for sharing!

dylz 11y ago

Damn. Was hoping for a self hosted (and full featured) disqus alternative or something to appear one day.

jonhmchan 11y ago
Discourse was really, really close for us. The breaker was the lack of a real WordPress import support: https://meta.discourse.org/t/is-it-possibile-to-import-existing-wordpress-comments-along-with-post-into-discourse/18157 https://meta.discourse.org/t/is-it-possibile-to-import-exist... That being said, that evaluation was done before we found out how difficult it would be with Disqus.

cddotdotslash 11y ago

I'm all for spending engineering time on non-project related tasks occasionally, but this seems like an excessive use of valuable time to reinvent the wheel. I'm sure they could have found a blogging platform that didn't require custom markdown interfaces, Python scripting, and re-writing everything from scratch. It's just a blog.

jacques_chester 11y ago
An old unix greybeard of my acquaintance told me circa 2004 that "writing a blog engine is this generation's version of writing a shell". Creation is more fun than selection. And in the same position, I'd ditch Wordpress like a stolen sports car.
hayksaakian 11y ago
I disagree. It seems like the biggest time-wasters were exporting all their data from wordpress without breaking anything. Even if they found "a blogging platform that iddn't require custom markdown interfaces" that would not have solved the problems they faced getting data out of wordpress.
- hueving 11y ago
  So why didn't they just fix WordPress. If export was expensive and they were willing to burn engineering time on it, the solution seems obvious. However, WordPress is not cool so they went to the tired old "reinvent blogging". No open source contributions, just building castles in their own sandbox.
  - jacques_chester 11y ago
    Wordpress is filled with third rails.
  - danso 11y ago
    I don't want to make this sound like an appeal to authority...but have you created/maintained both WordPress and static-generated blogs (either Jekyll, or something similar like Middleman)? Without having done both, I'd tend to agree with you: a blog is a blog. The opportunity cost from system migration (and re-invention) is non-trivial. But so are the opportunities gained with a new infrastructure. The speed gains from serving static files is nice, but to me, they are no where near the kind of gains I make by being able to work with plain text files. When I want to write a file, I pop open my (probably already open) text editor, bang it out, save, switch to Terminal, push to S3. Did I make a typo? OK, fix the typo, save, re-run the Terminal command. Maybe I want to push a few posts, but I want to make sure they all look right...OK, run the jekyll server and look at localhost. Then run the push command. Oh wait, maybe one of those posts should be a draft. OK, set it to draft in the metadata, re-push, etc. etc. I have a particularly bad experience with WordPress because I've been running my blog on it for five years on the cheapest Dreamhost shared server. However, I've spent a good amount of time researching and installing WordPress plugins to do caching...but the problem is that every goddamnned database action, including setting a post from draft to publish, or vice versa, is a 5 to 50 second process. Obviously, Stack Exchange doesn't suffer under that kind of hosting issue. And they could probably set up their WordPress on a local instance so that it can be locally browsed (in my experience, this can be profoundly difficult, and quite the weekend project). But no matter what their hosting infrastructure is, it still doesn't negate the fact that every posting action requires a database -- everything from creating users, setting user roles, setting post status, and of course, editing the post. Sure, you have to learn those things in Jekyll too...but the conventions are easy...and most importantly, you can do them within the confines of your text editor. That means everyone who contributes to the blog, engineers and marketers, gets better at their text editor (and operating system). This is a much more worthwhile skill than getting better at WordPress...which is a non-trivial skill...WordPress, to its credit, is fast moving in its development...for occasional publishers like me, this means I spent a good amount of cognitive energy re-learning the interface. With Jekyll, even as the software changes, I just have to learn my text editor. And that's just wonderful, considering how much of my work involves a text editor. For non-techies, I imagine it's a little daunting at first...but they're fully-functioning adults. They can handle plain text. And working with Jekyll not only makes it easier for the engineers to do the occasional hacking of templates, it allows the non-techies to actually _see_ it without having to jump through the many plugin submenus in the WordPress admin. It is literally a Cmd-T (in Sublime Text) to look up a file. Or, a Find All across a project...which is another huge benefit...blog contributors often need to look up past material...the text editor search is far superior to what can be done in the WP Admin or other web search interfaces. So it's not that "WordPress is not cool"...and the fact that you think that one solution is to "just fix WordPress" makes me think you don't understand the problem. WordPress doesn't need to be "fixed" to be more like Jekyll, it is a fundamentally different product, with certain tradeoffs and benefits worth pursing. To say that the OP should've "just fixed WordPress" is like saying, "Why use Facebook? Just get better at using the CC: field in email"

jlarocco 11y ago

It seems like this was a big waste of time reinventing Octopress.

tarr11 11y ago
Octopress is very difficult to use, and I would not recommend it. Using jekyll directly is a much better choice.
- jacquesm 11y ago
  Octopress is built on top of jekyll and what is a real nuisance is how long it takes to re-generate a site with more than a few posts.
  - Nick-Craver 11y ago
    This isn't true anymore. Octopress uses an older version of Jekyll. If you're on 3.0 (which we're using) has incremental generation. Here's the log of a recent incremental build: [Step 1/3] Jekyll build (11s) [20:52:40][Jekyll build] Jekyll build --config '_blog.config.yml' --no-watch [20:52:43][Jekyll build] Configuration file: C:/BuildAgent/work/4e27a39603eb2bf1/blog/_config.yml [20:52:43][Jekyll build] Source: C:/BuildAgent/work/4e27a39603eb2bf1/blog [20:52:43][Jekyll build] Destination: C:/BuildAgent/work/4e27a39603eb2bf1/blog/_site [20:52:43][Jekyll build] Incremental build: disabled [20:52:43][Jekyll build] Generating... [20:52:52][Jekyll build] done in 9.082 seconds.
    - jacquesm 11y ago
      Cool, time to upgrade then!

misterjinx 11y ago

It is not possible to subscribe to feeds individually ? For instance I would like to subscribe only for the engineering RSS feed. At the moment all I could find was a general feed for the entire blog.

vlad 11y ago

That’s exactly what I just did for my daily video / podcasting site Rate My App.com. - Everything in markdown - Generate the podcasts as an RSS feed - Generate the list of videos as JSON, filtered inside the AngularJS app Such an approach is not great for SEO in the short-term, so I won't be surprised if they lose ranking because their posts and comments are missing from the the html until their jQuery code fetches and renders them. For me, I don't have 700 posts so that's easy to address later once I have content worth indexing, keeping the urls the same of course. :)

chaselee 11y ago
Check out [prerender](https://prerender.io/ https://prerender.io/) to get around the SEO issue.

kmfrk 11y ago

Static blogs are doing pretty well, but setting up a sepparate interface or flow for writers - not admins and developers - is still a big of a chore, unless you don’t mind giving people full access to the site repo. I’m thinking about setting up a separate draft and post repo and importing them into the root with git subtree.

philippnagel 11y ago

Are there any good, corporate solutions for organizations that lack in-house engineering resources?

saosebastiao 11y ago
Jekyll is pretty low impact as far as engineering goes...all the engineering work here went into migration of the existing blog. I would think getting Jekyll set up would be even easier than a self-hosted Wordpress installation.

rebootthesystem 11y ago

They had me at "Not Wordpress or PHP"