High quality Python scripts or small libraries to learn from

I think I mention this all the time when this comes up, but I learned the most 'best practices' through using ruff.

https://docs.astral.sh/ruff/

I just installed and enabled all the rules by setting select = [ "ALL" ]

And then looked at the 'errors' it showed me for my existing code base and then excluded some rules which don't interest me.

I also have it set up in my IDE, so it'll show me the linting errors while coding.

In a larger code base there will definitely be many 'errors', but using it cleaned up my code really good and it stopped me from some footguns.

Simon Willison's github would be a great place to get started imo -

https://github.com/simonw/datasette https://github.com/simonw/sqlite-utils

So, his code might not be a good place to find best patterns (for ex, I don't think they are fully typed), but his repos are very pragmatic, and his development process is super insightful (well documented PRs for personal repos!). Best part, he blogs about every non-trivial update, so you get all the context!

At my university, I followed David Beazly's talks and tutorials. Just seeing him work and present improved my style and approach manifolds:

https://www.dabeaz.com/tutorials.html

I think I've learned more reading bad code bases than reading good code bases.

The entire point is not to just mindlessly consume a code base, but instead form an idea of how to approach the problem and then see if your hypothesis is correct. Then comparing your approach to the actual approach.

This can show you things that you might've missed taking into account.

For example, gallery-dl's incidental complexity all lies in centralizing persistent state, logging, and IO through the CLI. It doesn't have sufficient abstraction to allow it to be rewired to different UIs without relying on internal APIs that have no guarantee that won't change.

Meanwhile a similar application in yt-dlp has that abstraction and works better, but has similar complexity in the configuration side of things.

I can't think of any (small) libraries I could recommend to learn best practices, but what does come to mind is click [0], the CLI library for Python. Their documentation is pretty great, there are tons of short example scripts to be found online, and in my experience making little apps with click can be a nice way to learn different python features like args/kwargs, decorators, string manipulation, etc.

I agree with others that code formatters like black or ruff might be helpful to you. The literature surrounding them, such as PEPs concerning code formatting, often include examples you may find useful.

[0] https://click.palletsprojects.com/en/8.1.x/

If you're looking for some best practices related but limited to machine learning application code, you could have a look at Beyond Jupyter (https://github.com/aai-institute/beyond-jupyter)

Here's an excerpt from the readme: "Beyond Jupyter is a collection of self-study materials on software design, with a specific focus on machine learning applications, which demonstrates how sound software design can accelerate both development and experimentation."

For scripts, I've learned a couple tricks from OpenAI's Cookbook examples.

This one came in handy not too long ago: https://github.com/openai/openai-cookbook/blob/main/examples...

Peter Norvig's work is great to learn from https://github.com/norvig/pytudes

Pyupgrade is a good tool that focuses on upgrading to newer idioms. Which is more important for learning than simple pep8 type stuff, which is useful but has its limits.

On that subject, Raymond Hettinger has a great talk called “Beyond Pep8” that talks about how to de-java your codebase among other things. Also reading the book Fluent Python now and it is so far excellent.

Take a look at PY4WEB: https://github.com/web2py/py4web

It is an improvement for the the tiny but efficient web2py web framework.

You should definitely read bottle.py, while full of hack to support python2 it still a very good code base to learn about many python features. Another one is stencil template engine.

Everything @simonw has worked on, honestly: https://github.com/simonw

I don't know if I can consider my code "Great" but I dedicated way too many months on a prometheus library where I focused on quality since I did it for me.

It's relatively small and I think the main take away would be the use of Protocols for the pluggable backend system. I hope you get something out of it :)

https://github.com/Llandy3d/pytheus

You do what everybody does:

Grind this out painstakingly and then copy and paste that as boilerplate for the rest of your natural life.

Karpathy also writes beautiful python code and his GPT lecture series contains some of the finest python code to learn from. https://github.com/karpathy/nn-zero-to-hero

I've recently looked at tasktiger https://github.com/closeio/tasktiger. It's a simple queue system that helped me understand how workers and schedulers work.

I'd suggest Flask or some of the smaller projects in the Pallets ecosystem:

Flask, in particular, has a very small number of open issues (2) for a project that is pretty popular. Its also maintained by a competent team and has a lot of project best practices.

https://github.com/pallets/flask

https://github.com/pallets

https://pypistats.org/packages/flask

https://pypistats.org/packages/django (for comparison)

Mostly you should start by reading what's shipped with core. After that, look at larger projects with lots of committers, as they often end up getting "polished" down to something that's sort of generically useful without being too quirky or too simple, and the architecture tends to be more functional.

Avoid anything developed/maintained by one corporation. In general, their organizational hierarchy leads to bad patterns and resists useful changes that don't conform to the goals and patterns of the business or engineering leads. Grassroots OSS projects aren't always better, but they're less likely to have a monoculture and perverse incentives.

The Django code base is excellent. I learned a ton early on by reading through it

The 500 Lines or Less section from https://aosabook.org/en/

The Architecture of Open Source Applications

Peter Norvig’s python scripts are quite beautiful.

I had 2 suggestions (plus a blog post) in my style guide here:

https://github.com/amontalenti/elements-of-python-style#some...

The style guide itself, published a few years back, also has some suggestions with small code snippets.

Check out https://github.com/recursion-computing/starcel-panda3D for bleeding edge, but not refactored to be pythonic, Python OS development.

if you are into ML libraries, take a look at fklearn, a scikitlearn-like lib, but written in a functional. Fun read to compare both side by side.

Any codebase that uses and respects type annotations is probably a good place to start. So grep for "from typing".

SQLAlchemy sources.

Not exactly small, but great.

The python requests library

just try to build stuff and then u will see over time what works and what causes issues. theres no shortcuts other than using chatgpt

what’s your experience level? what kind of python code do you work with (web dev? Data analysis? Algorithms?)