Tried this on our project, GitHub.com/hail-is/hail.
Sphinx is a persistent thorn in our side, but we've made it work. A couple things I had to do to get this working:
- something thinks a filename with dots (e.g. example.8bits.bgen) means there is a python module that needs to be imported (e.g. example), and fails when that does not exist; we have files like this in a data directory
- I could not quickly figure out how to ignore files, so I had to delete a Sphinx conf.py file.
- Sphinx resolves the _templates folder relative to where it is called, not relative to the doc string's source file, so I had to add several symlinks (rather than manually edit every doc string).
---
OK docs are generating. Quite slow, maybe a couple minutes. Looks like one core is at 100% during this process. We've got ~400 files. Maybe its parsing some data files? The whole directory is 318 MB.
---
Now I'm looking at the docs.
First thing I notice is that Python type hint forward references break everything. Consider:
class GroupedMatrixTable(
parent: 'MatrixTable',
...
The generated doc page is missing the class name and the entire class definition is inlined as a preformatted block (with highlighting :shrug:).
All my python function def's seem to be missing the function name?
No arg functions look really weird
def (
)
So all our docs are in ReST (thanks Sphinx :|). This means they're not valid Markdown. It seems that invalid markdown in a class doc string can break the formatting of all the methods (presenting them again as one reformatted block).
---
On the bright side, the mobile version looks great and the search works better than my experience with Sphinx.
Thanks so much for this great feedback! I haven't done any speed optimization on portray yet - as both at work and online I tend to organize my own projects as small self contained repositories. Clearly there is a lot of room for improvement there, I have a lot of ideas to make it faster.
And, yes write now Markdown only. I will say - this was a week only project: https://timothycrosley.com/project-2-portray so I think there's a good chance I could improve these points with one more week of time spent :)
Yeah of course! In retrospect my initial post seems a bit neutral to negative. I'm really glad this project exists, I'm endlessly frustrated by Sphinx. I'll keep my eye on your project!
A little off-topic, but as someone who's just started producing documentation in Sphinx, what are the problems/downsides that you've run in to?
I've used it for two projects so far: documenting a relatively small python module, and for documenting a RESTful API. I do a lot of embedded work, so I'd like to use it as a place to document a piece of hardware in it's entirety, from schematics/layouts to firmware and build toolchains to actual use of the device. Do you (or anyone else reading this) have any comments on that use case?
I think my main complaint at the moment is that it doesn't play nice with Markdown, so I had to re-format a bunch of pre-existing documentation for it to work in Sphinx. ReST seems to render alright in Gitlab though, so that's a plus.
My team at present use sphinx-doc[1] and generate both HTML and PDF. We write the documentation as part of the code using docstring and then other documents manually in rst. We use Sphinx plugin to automatically generate document from code.
I will give portray a try and see.
Is there any comparison between Sphinx-doc and portray?
We've historically done this as well. One issue we perpetually run into is that the docstrings get out of sync with the actual signature / types of the methods/etc that they purport to document. I suspect there are plugins that generate documentation from types such that there is less to keep synchronized. It is also unfortunate that Python projects have to set up infrastructure to build and publish documentation packages. I really like the effortlessness that Go's ecosystem brings--https://godoc.org just reads your source code from your git repository, so there is are no explicit steps for building and publishing documentation.
With Sphinx's Napoleon extension (https://www.sphinx-doc.org/en/master/usage/extensions/napole...) Python 3 type annotations will be added to the generated docstrings so you don't have to write the annotations both in the function definition and in its docstring. I've found this to be very helpful in keeping docs in sync with the code.
It's kind of on the whole dynamically typed model. If you can have static type annotations that are validated for correctness, then you can use those annotations to validate or generate your documentation. Python supports annotations and validation thereof via Mypy, and assuming the project takes advantage of these, then Sphinx could leverage them to validate/generate documentation.
I use sphinx-apidoc[1] command to extract docstrings from code and create Sphinx documentation files out of them.
Enter this command to look for modules in a package, say, foopkg, and create .rst files to generate documentation for each module in the package and subpackages:
sphinx-apidoc --module-first -o docs/api foopkg
I am assuming that the Sphinx documentation project resides in a directory named docs. Each .rst file would contain automodule directives for Sphinx to automatically pull docstrings from foopkg and render them in the generated documentation. However, for Sphinx to understand these automodule directives, the autodoc[2] extension must be enabled in docs/conf.py.
extensions = ['sphinx.ext.autodoc']
If the documentation is written using Google style or NumPy style docstrings, then the napolean[3] extension must also be enabled.
The sphinx-apidoc command above would also write a file named docs/api/modules.rst which would be used to render the starting page of the automatically generated API documentation. So include this docs/api/modules.rst with the Sphinx toctree directive from the some other page that the user is likely to visit. For example, the start page of the project documentation would likely be rendered from `docs/index.rst`, so add this to `docs/index.rst`:
API
---
.. toctree::
api/modules
Now render the documentation with Sphinx normally:
This is great! You hit the need for configless docsites that many Pythonistas who are in data science and research have: we write a lot of experimental and prototype code. Docstring documentation in itself feels like a considerable investment in these contexts so anything that helps publication without any hassle is more than welcome.
Are you planning to support Restructured Text next to Markdown? I find ResT is used more often (in the machine learning ecosystem at least).
I just used this on a random project I happen to be building documentation for. I'm pretty in love with the minimalism of 'portray'. I wish there was a chance to get my team to use it, but I'll definitely be using it in my personal projects going forward!
portray requires your project to be packaged for it's auto discovery - if you get this error and your project has a setup.py or pyproject.yaml in the root (and this is where you ran portray from) let me know and I'll create a ticket! Otherwise - I highly encourage looking at using poetry to setup your pyproject.yaml file: https://poetry.eustace.io/
my project doesn't have a setup.py because it is not intended for packaging and it doesn't have a pyproject.yaml because to be honest, this is the first time I heard about such a file. But then again, the title says "zero-config".
I saw that as well, late into the project creation. The pdoc3 maintainer claims it's not that - but there are other reasons to want to want to avoid pdoc3. I'm going to be working on replace the pdoc3 dependency with pdoc in a way that is transparent to users.
Great to see you are receptive to this change. I didn't notice it earlier but then when I see on the right hand corner of the website I was astonished it's there.
There is no question that these are Buddhist swastikas; they are clockwise and also rotated 45 degrees compared to the most common nazi version.
It is difficult to appreciate how ubiquitous swastikas are in the Buddhist world until you go there and see them everywhere. I have no doubt that this was an innocent decision.
It stopped being an innocent decision once it was pointed out and he doubled down.
Put another way: I cannot put my or my employer's interests in Western markets at risk because something is acceptable in Eastern markets. The maintainer can do whatever he wants to do; I'm not questioning that. I'm just saying I won't be using it, even if it would save me from sphinx-autodoc fragility.
As the original author of pdoc, thanks for not using pdoc3, which---even separate from the Nazi imagery---is something that I consider to be an unethical fork. You might instead consider using the original pdoc, which is now maintained by mitmproxy[1]. Although, it does seem like activity on the project has waned.
> I cannot put my or my employer's interests in Western markets at risk because something is acceptable in Eastern markets
My employer works in both markets, so rejecting something because it's Eastern would look much worse than rejecting it because someone in one market might not know what it is.
Anyway, it's a tiny icon on the boiler plate of a dependency of a project used to generate documentation. It would be a huge stretch to say your employer supports Nazis because of this.
I'd be happy not seeing references to any religion and, quite frankly, I would hesitate to associate my employer with anything espousing religious dogma.
Fuck all the downvoters, im not +1-ing Nazis... they fucking stole that symbol!
Its thousands of years old. Nazis have a bad name because they were fucking nazis!
The swastika is an ancient Sanskrit symbol which represents the churn of time and and the cycle of the universe.
Its really interesting to think about how the symbol has been coopted to be edited out of human conscience such that we dont recognize its symbology for the cycle of time and now its only associated with WWII
People are so stupid with symbology it boggles my mind:
Source: mayan order centurion, masons, rosicrucian.
Wait — you want people to stop “editing” history and then want to go back to a version of the symbol’s history before the holocaust?
Hint: if you change history by appending to it, it is not changing history it is writing it.
If you look back into history and check what happend, that is reading history.
If you want to delete lines that don’t fit your world view from the history, that is editing.
How would you go back to the original meaning of the swatsika without denying the existence of Nazi Germany and what it did?
You can stop giving the Nazis so much influence over your life, for starters. Just accept that they, like plenty of societies before them, used a certain cool logo, but that's no reason to hate that logo, or to never use it again.
I grew up in a Austrian Province with a 40% rightwing majority, most of which hailed Hitler after a few beers and one of my grandfathers was a convinced nazi.
Be convinced I thought enough about how much influence nazis have about my life, be it direct or by the positions I take to oppose them. And yeah: Nazis don’t go away of you don’t talk to them or ignore their existence. Just because you tolerate them won’t mean they will tolerate you.
And on the symbol: the swatsika in western culture is forever stained. This might be different if we were a grown up society that managed to deal properly with the past — but you can be sure that this week some young neo nazi tatoos the swatsika on their skin as a symbol of their racial hate — and while that is still going on a simple redefinition (that neo nazis would love to see), is frankly speaking naive.
Not sure if you're accidentally excluding the >1 billion Hindus (and Jains) from your list or if that was an intentional dog whistle towards Indians and other South Asians.
I'll assume it's the former.
The reason why I'll assume it's the former is because we should always assume good faith - it's even a posting guideline on HN[0].
It was accidental, but I also didn’t really intend to give an exhaustive list of uses here.
Also we are talking about a swatsika beeing used with near certainty in a western context. Maybe my reaction was indeed a little too “alergic” — I grew up in Austria and I have memories of Nazi grandfathers giving their neonazi ancestors all those nazi insignia. People who after the 10th beer start praising Hitler and demand new genozides, once they are in power again.
Given my background seeing somebody claiming their swatsika is not linked to nazis because it faces the wrong way is just something that doesn’t fly — I have heared that (and similar) excuses far too often from people who were definitly convinced nazis.
Btw. I don’t know if that guy is a nazi or someone who wants to provoke, but he certainly seems to know about the difference between right and left facing swatsika and thus could’ve anticipated the whole discussion.
His github profile picture is the swastika counter-clockwise. It's very clear from that his intention is to include things that appear like the Nazi swastika but are technically not. That's bad intention in my book.
In all fairness to the GitHub account, the Python logo itself could be perceived as the same logo made out of two snakes.
But troll culture masquerading as innocent plausible-deniability really is ruining the internet because it's hard not to assume the worst (we see the same symbols being used on purpose to illicit a reaction and that person did respond with "Made you look. :grin:").
It's true that that swastika (usually going in the other direction) is a buddhist symbol, but it looks like the person who forked it is from Slovenia, which has never been a buddhist country but was briefly a Nazi country. It's hard to believe that was an accidental oversight.
I think being told what you are not allowed to do by strangers on the internet can elicit a certain stubbornness. I don't blame either side, but think it is strange for people to be so sure of another's intentions.
For what it's worth, I'm not claiming I'm certain he's a nazi sympathizer. What I am claiming is I cannot risk media or competitors making a big deal out of it, even if he means well. Surely that isn't a far out proposition, right?
Symbols get irrevocably besmirched all the time. I'm not going to be the person roping in a documentation tool that results in "<FAANG Company X> uses software linked to Nazis" as a BBC headline, you know? That headline alone is going to roflstomp any amount of nuanced rhetoric - and that's giving the maintainer the absolute most kind benefit of the doubt, which I'm not even remotely convinced they actually deserve.
It's still in the footer. From the screenshot I thought it was directly around the quote but it's actually closer to the edges of the screen when in desktop view.
My bad. I was looking in the wrong spot, they are indeed all the way out to the side and pretty light. Probably in my blind spot on this particular screen size!
I thought it's common knowledge the Nazi symbol is rotated 45° and vents clockwise. Still, agree that removing it is the smart move to increase adoption.
Many comments are addressing the fact that there is a difference between the clockwise and counter-clockwise symbols and that they should not be confused, but they are both on the website?
It is quite apparent that he knows about the meanings and uses of the symbol, therefore he could (and should) have anticipated the discussion. I grew up in an environment where nazi insignia are forbidden, and a very common way for convinced nazis to get around this was to just flip the damn thing and claim it is ok now, while in spirit you still go and use it to hail hitler with your friends.
I grew up in southern Austria next to the Slovenian border (which is where this guy is based). This part of Austria had a 40% right wing majority in the past and after a few beers right wingers still openly hail hitler in a pub, despite that beeing illegal. My grandfather who passed away a few years ago was a convinced nazi — this thing is still alive here.
The author is based in slovenia, literally a hour by car from where I lived. There is no way somebody from that area uses that symbol accidentally, without having considered its nazi connotations. This is an area where every other person had somebody in his family killed by nazis or were profiting from it in some way.
So either this person is incredibly naive and tries to overwrite history that is still very much ongoing or it is somebody who in full knowledge of all of this still decided to go with the symbol. In any way it doesn’t look good to me.
You know the Python symbol is a swastika, right? Technically it's a couple snakes that are coiled like a swastika, but if it's "anything to do with a swastika" we should avoid it, right?
I think the average person is going to have to have you point out the relation and squint for a while before they see it, if at all. I had to close one eye and really look at it before I saw what you were talking about.
Sphinx is a persistent thorn in our side, but we've made it work. A couple things I had to do to get this working:
- something thinks a filename with dots (e.g. example.8bits.bgen) means there is a python module that needs to be imported (e.g. example), and fails when that does not exist; we have files like this in a data directory
- I could not quickly figure out how to ignore files, so I had to delete a Sphinx conf.py file.
- Sphinx resolves the _templates folder relative to where it is called, not relative to the doc string's source file, so I had to add several symlinks (rather than manually edit every doc string).
---
OK docs are generating. Quite slow, maybe a couple minutes. Looks like one core is at 100% during this process. We've got ~400 files. Maybe its parsing some data files? The whole directory is 318 MB.
---
Now I'm looking at the docs.
First thing I notice is that Python type hint forward references break everything. Consider:
The generated doc page is missing the class name and the entire class definition is inlined as a preformatted block (with highlighting :shrug:).All my python function def's seem to be missing the function name?
No arg functions look really weird
So all our docs are in ReST (thanks Sphinx :|). This means they're not valid Markdown. It seems that invalid markdown in a class doc string can break the formatting of all the methods (presenting them again as one reformatted block).---
On the bright side, the mobile version looks great and the search works better than my experience with Sphinx.
EDIT: formatting