6 ms·

Show HN: A tool to convert Jupyter notebooks to beautiful blogs

150 points by hemanta212 7y ago

the_watcher 7y ago
Neat! Anyone working on projects to improve the share-ability, code review, version control, or web rendering of Jupyter notebooks, feel free to reach out to me when looking for an enthusiastic early user. Jupyter has been a game changer for my productivity as a data scientist, but I still struggle with something that seems as simple as "what's the best way to expose this to other people easily without having to set up a self-hosted nbviewer".
- amrrs 7y ago
  Is it tough to manually convert the md and use a CMS like Ghost or Hugo for website? Alternatively, exporting HTML and publishing can also help right? I'm curious why did you say so. Moreover did you check the recently released `voila` package that lets you publish Jupyter notebooks?
  - the_watcher 7y ago
    There are a bunch of ways that make it _possible_ to do all of this. What's really missing is a way to make it as easy as committing code to Github repo without having to do anything more than install a Github extension. Right now, I do my work in a notebook, it's automatically converted to a Markdown file (for code review), and I use nbconvert to generate static html, which gets deployed to an internal docs site. I've also been tinkering with ReviewNB, which is trying to make notebook review look like code review, but it's separate from Github and is still fairly unpolished (but very promising!). Ideally, the experience could be something like: * Install Github extension that offers a notebook review tab and automatically converts notebooks to a static HTML file that can be served from a url directly accessible from Github I can think of a bunch of challenges involved in doing something like the above, but my experience working somewhere that offered a tool that allowed me to literally click one button within my notebook and have a shareable url that contained my notebook and the ability to comment line-by-line taught me that this massively unblocks data scientists, who will otherwise end up copy/pasting stuff into documents just to deliver things to partners faster.
    - amirathi 7y ago
      > and is still fairly unpolished (but very promising!). I'm all ears. Please write to me at amit@reviewnb.com I built ReviewNB and push improvements almost every single day :)
  - the_watcher 7y ago
    By the way, voila looks very promising as well, thanks for the pointer.
- lhnz 7y ago
  Two projects I've been watching with interest are https://kyso.io/for-teams https://kyso.io/for-teams and http://reviewnb.com http://reviewnb.com.
  - the_watcher 7y ago
    ReviewNB is very promising! I just wish it could live within Github (my idea is something like how Zenhub reorganizes issues into a Kanban board - add a tab that only allows viewing of ipynb files), since right now I have to kind of cajole people to review notebooks on their site.
    - amirathi 7y ago
      I am the developer behind ReviewNB. > I just wish it could live within Github Whenever I thought of integrating the review experience in the GitHub UI (via browser extension), I always thought of overwriting the GitHub diff UI. That approach is very hacky and would result in the extension stability issues (any DOM change in GitHub diff could break the extension). But I just looked at Zenhub and having a separate tab for notebook review could actually work! Since it relies only on the tab related HTML, the extension would be relatively stable. There's still a question of how to post comments etc. But it might be a good start to just offer notebook diff in a separate GitHub tab for pull requests. Thanks @the_watcher.
      - the_watcher 7y ago
        Glad you saw this!
- AlexSolution 7y ago
  https://github.com/mwouts/jupytext https://github.com/mwouts/jupytext looks like something you might enjoy. I'm not affiliated with the project in any way.
- dbecker 7y ago
  If you send me an email (dan@kaggle.com) I'd love to set up a time to show you some mockups that may be the solution you are looking for.
- adsodemelk 7y ago
  We are developing a Notebook like interface for SQL https://uktechdata.count.co/ https://uktechdata.count.co/ https://stackoverflow.count.co/ https://stackoverflow.count.co/ Welcome to get your thoughts on this.
fifnir 7y ago
Nice, have you looked into jupyter-themes? I've played with it a bit to make a similar notebooks-->static project and it's really great!
madenine 7y ago
Cool, but I would rather use a better supported static site generator with an ipynb plugin (ie, https://github.com/danielfrg/pelican-ipynb https://github.com/danielfrg/pelican-ipynb) than a tool just for notebooks. Especially because this looks like an nbconvert workflow under the hood.
- hemanta212 7y ago
  > Especially because this looks like an nbconvert workflow under the hood. It is, however I convert it to the most basic html and add custom css and js to it. So that I can change it to whatever I want. I don't see any problem with that maybe you could point some out. I didn't go with pelican(or any static gen) because the build times will be much slow later on as blogs increase [1] as this guy faced. I like to configure a lot of things manually and don't want the burden of static gen if that make sense to you. I just needed a converter. 1. https://nipunbatra.github.io/blog/2017/Jupyter-powered-blog.html#Other-fixes https://nipunbatra.github.io/blog/2017/Jupyter-powered-blog....
petters 7y ago
I have never understood why the "In[5]" and "Out[5]" in jupyter have to be so prominent, even in a output like this.
- t1amat 7y ago
  It seems useful to denote REPL input vs output, and to be able to identify output relating to input.
  - petters 7y ago
    Sure, but the [5] feels unnecessary in a finished document like this. I mean, the output should be right after the input anyway, right?
    - mkl 7y ago
      It's so it's easy to refer to things, like equation numbers in a maths book, or line numbers in code. E.g. people talking about someone's notebook here on HN can be specific and clear.
- smcmurtry 7y ago
  The numbers track the order that the cells were executed in. So, strictly speaking, you need the numbers to be able to reproduce the output of the notebook's author. However, I think it's bad practice and pretty uncommon to publish a notebook where you need to execute the cells out of order. So most of the time the numbers aren't useful.
  - analog31 7y ago
    Indeed, and the line numbers are a warning that you might have an order-of-execution bug lurking somewhere.
- lallysingh 7y ago
  Agreed. They should be (5) Input: / Output:. Where input and output are on the same level of indentation under (5).
tluyben2 7y ago
Offtopic; does anyone know a tool that can turn a directory of source code into a nicely formatted PDF/PS? So with syntax highlighting etc based on the file you have and chapter index based on directory structure for instance?
- jason_slack 7y ago
  Pandoc? I haven’t used it in a while though.
bufferoverflow 7y ago
Did something go wrong here? https://pykancha.github.io/test/2019/selenium_p1.html https://pykancha.github.io/test/2019/selenium_p1.html
- hemanta212 7y ago
  Nope. It was to test my native language for unicode support.
avilay 7y ago
Nice. I have been using gitlab CI and a simple Python script to publish my notes (usually in markdown and Jupyter notebooks) to Gitlab pages. You can find the output on http://avilay.gitlab.io/gyan http://avilay.gitlab.io/gyan. And if you want to reuse the code you can find it in https://gitlab.com/avilay/gyan/blob/master/publisher/publish.py https://gitlab.com/avilay/gyan/blob/master/publisher/publish.... Will surely check out your stuff.
- hemanta212 7y ago
  I didn't find it responsive though. Maybe not your goal but the designs are similar. Similarly I think you can import nbconvert instead of just calling from subprocess.
russfink 7y ago
I browsed the example link given at the GitHub site, and found this blog entry on lessons to tell your younger self. This post is amazing! All the advice rings true. Worth a read! https://pykancha.github.io/test/2018/cs-phd-lessons.html https://pykancha.github.io/test/2018/cs-phd-lessons.html
- hemanta212 7y ago
  Here is original link: https://nipunbatra.github.io/blog/2018/cs-phd-lessons.html https://nipunbatra.github.io/blog/2018/cs-phd-lessons.html

150 points by hemanta212 7y ago

Neat! Anyone working on projects to improve the share-ability, code review, version control, or web rendering of Jupyter notebooks, feel free to reach out to me when looking for an enthusiastic early user. Jupyter has been a game changer for my productivity as a data scientist, but I still struggle with something that seems as simple as "what's the best way to expose this to other people easily without having to set up a self-hosted nbviewer".

amrrs 7y ago
Is it tough to manually convert the md and use a CMS like Ghost or Hugo for website? Alternatively, exporting HTML and publishing can also help right? I'm curious why did you say so. Moreover did you check the recently released `voila` package that lets you publish Jupyter notebooks?
- the_watcher 7y ago
  There are a bunch of ways that make it _possible_ to do all of this. What's really missing is a way to make it as easy as committing code to Github repo without having to do anything more than install a Github extension. Right now, I do my work in a notebook, it's automatically converted to a Markdown file (for code review), and I use nbconvert to generate static html, which gets deployed to an internal docs site. I've also been tinkering with ReviewNB, which is trying to make notebook review look like code review, but it's separate from Github and is still fairly unpolished (but very promising!). Ideally, the experience could be something like: * Install Github extension that offers a notebook review tab and automatically converts notebooks to a static HTML file that can be served from a url directly accessible from Github I can think of a bunch of challenges involved in doing something like the above, but my experience working somewhere that offered a tool that allowed me to literally click one button within my notebook and have a shareable url that contained my notebook and the ability to comment line-by-line taught me that this massively unblocks data scientists, who will otherwise end up copy/pasting stuff into documents just to deliver things to partners faster.
  - amirathi 7y ago
    > and is still fairly unpolished (but very promising!). I'm all ears. Please write to me at amit@reviewnb.com I built ReviewNB and push improvements almost every single day :)
- the_watcher 7y ago
  By the way, voila looks very promising as well, thanks for the pointer.
lhnz 7y ago
Two projects I've been watching with interest are https://kyso.io/for-teams https://kyso.io/for-teams and http://reviewnb.com http://reviewnb.com.
- the_watcher 7y ago
  ReviewNB is very promising! I just wish it could live within Github (my idea is something like how Zenhub reorganizes issues into a Kanban board - add a tab that only allows viewing of ipynb files), since right now I have to kind of cajole people to review notebooks on their site.
  - amirathi 7y ago
    I am the developer behind ReviewNB. > I just wish it could live within Github Whenever I thought of integrating the review experience in the GitHub UI (via browser extension), I always thought of overwriting the GitHub diff UI. That approach is very hacky and would result in the extension stability issues (any DOM change in GitHub diff could break the extension). But I just looked at Zenhub and having a separate tab for notebook review could actually work! Since it relies only on the tab related HTML, the extension would be relatively stable. There's still a question of how to post comments etc. But it might be a good start to just offer notebook diff in a separate GitHub tab for pull requests. Thanks @the_watcher.
    - the_watcher 7y ago
      Glad you saw this!
AlexSolution 7y ago
https://github.com/mwouts/jupytext https://github.com/mwouts/jupytext looks like something you might enjoy. I'm not affiliated with the project in any way.
dbecker 7y ago
If you send me an email (dan@kaggle.com) I'd love to set up a time to show you some mockups that may be the solution you are looking for.
adsodemelk 7y ago
We are developing a Notebook like interface for SQL https://uktechdata.count.co/ https://uktechdata.count.co/ https://stackoverflow.count.co/ https://stackoverflow.count.co/ Welcome to get your thoughts on this.

fifnir 7y ago

Nice, have you looked into jupyter-themes? I've played with it a bit to make a similar notebooks-->static project and it's really great!

madenine 7y ago

Cool, but I would rather use a better supported static site generator with an ipynb plugin (ie, https://github.com/danielfrg/pelican-ipynb https://github.com/danielfrg/pelican-ipynb) than a tool just for notebooks. Especially because this looks like an nbconvert workflow under the hood.

hemanta212 7y ago
> Especially because this looks like an nbconvert workflow under the hood. It is, however I convert it to the most basic html and add custom css and js to it. So that I can change it to whatever I want. I don't see any problem with that maybe you could point some out. I didn't go with pelican(or any static gen) because the build times will be much slow later on as blogs increase [1] as this guy faced. I like to configure a lot of things manually and don't want the burden of static gen if that make sense to you. I just needed a converter. 1. https://nipunbatra.github.io/blog/2017/Jupyter-powered-blog.html#Other-fixes https://nipunbatra.github.io/blog/2017/Jupyter-powered-blog....

petters 7y ago

I have never understood why the "In[5]" and "Out[5]" in jupyter have to be so prominent, even in a output like this.

t1amat 7y ago
It seems useful to denote REPL input vs output, and to be able to identify output relating to input.
- petters 7y ago
  Sure, but the [5] feels unnecessary in a finished document like this. I mean, the output should be right after the input anyway, right?
  - mkl 7y ago
    It's so it's easy to refer to things, like equation numbers in a maths book, or line numbers in code. E.g. people talking about someone's notebook here on HN can be specific and clear.
smcmurtry 7y ago
The numbers track the order that the cells were executed in. So, strictly speaking, you need the numbers to be able to reproduce the output of the notebook's author. However, I think it's bad practice and pretty uncommon to publish a notebook where you need to execute the cells out of order. So most of the time the numbers aren't useful.
- analog31 7y ago
  Indeed, and the line numbers are a warning that you might have an order-of-execution bug lurking somewhere.
lallysingh 7y ago
Agreed. They should be (5) Input: / Output:. Where input and output are on the same level of indentation under (5).

tluyben2 7y ago

Offtopic; does anyone know a tool that can turn a directory of source code into a nicely formatted PDF/PS? So with syntax highlighting etc based on the file you have and chapter index based on directory structure for instance?

jason_slack 7y ago
Pandoc? I haven’t used it in a while though.

bufferoverflow 7y ago

Did something go wrong here? https://pykancha.github.io/test/2019/selenium_p1.html https://pykancha.github.io/test/2019/selenium_p1.html

hemanta212 7y ago
Nope. It was to test my native language for unicode support.

avilay 7y ago

Nice. I have been using gitlab CI and a simple Python script to publish my notes (usually in markdown and Jupyter notebooks) to Gitlab pages. You can find the output on http://avilay.gitlab.io/gyan http://avilay.gitlab.io/gyan. And if you want to reuse the code you can find it in https://gitlab.com/avilay/gyan/blob/master/publisher/publish.py https://gitlab.com/avilay/gyan/blob/master/publisher/publish.... Will surely check out your stuff.

hemanta212 7y ago
I didn't find it responsive though. Maybe not your goal but the designs are similar. Similarly I think you can import nbconvert instead of just calling from subprocess.

russfink 7y ago

I browsed the example link given at the GitHub site, and found this blog entry on lessons to tell your younger self. This post is amazing! All the advice rings true. Worth a read! https://pykancha.github.io/test/2018/cs-phd-lessons.html https://pykancha.github.io/test/2018/cs-phd-lessons.html

hemanta212 7y ago
Here is original link: https://nipunbatra.github.io/blog/2018/cs-phd-lessons.html https://nipunbatra.github.io/blog/2018/cs-phd-lessons.html