How I use Artipie, a PyPI repo

While developing with Python as a student, I found that I needed some private centralized storage. This was so I could store binary and text data files, as well as Python packages. I found the answer in Artipie, an open source self-hosted software repository manager.

At university, my colleagues and I conducted research and worked with a lot of data from experimental measurements. I used Python to process and visualize them. My university colleagues at the time were mathematicians and didn't have experience with software development techniques. They usually just passed data and code on a flash drive or over email. My efforts to introduce them to a versioning system like Git were unsuccessful.

Python repository

Artipie supports the PyPI repository, making it compatible with both twine and pip. This means you can work with the Artipie Python repository exactly as you would when installing or publishing packages on the PyPI and TestPyPI repositories.

To create your own Python repository, you can use the hosted instance of Artipie called Artipie Central. Once you sign in, you see a page with your repositories listed (which is empty to begin with) and a form to add a new repository. Choose a name for your new repository (for example, mypython), select "Python" as the repository type, and then click the Add button.

Next, you see a page with repository settings in the YAML format:

---
​repo:
  type: pypi
  storage: default
  permissions:
    olenagerasimova:
      - upload
    "*":
      - download

The type mapping in the configuration sets the repository type. In this example, the Python repository is configured with the default Artipie Central storage.

The storage mapping defines where all of the repository packages are stored. This can be any file system or S3 storage compatible location. Artipie Central has a preconfigured default storage that can be used for tests by anyone.

The permissions mapping allows uploads for the user olenagerasimova, and allows anyone to download any package.

To make sure this repository exists and works, open the index page in your browser. The packages list is displayed. If you've just created a new repository but have yet to upload a package, then the repository index page is blank.

Binary repository

You can store any kind of file in Artipie. The storage type is called file or binary, and I use this as storage for experimental data. I use this as input for Python visualizations. A file repository can be created in Artipie Central the same way as a Python repository. You give it a name, choose the type binary, and then click the Add button.

---
​repo:
  type: file
  storage: default
  permissions:
    olenagerasimova:
      - upload
      - download
    "*":
      - download

The settings are basically the same as for Python. Only the repository type differs. The binary repository, in this example, is called data. It contains three text files with some numbers:

​6
3.5
5
4
4.5
3
2.7
5
6
3
1.2
3.2
6

The other two files take the same form (only the numbers are different.) To see the files yourself, open the links one, two, and three in your browser and download the files, or you can perform a GET request using httpie:

​httpie -a https://central.artipie.com/olenagerasimova/data/y1.dat > ./data/y1.da

These files were uploaded to the Artipie Central data repository with PUT requests:

​httpie -a olenagerasimova:*** PUT https://central.artipie.com/olenagerasimova/data/y1.dat @data/y1.dat

httpie -a olenagerasimova:*** PUT https://central.artipie.com/olenagerasimova/data/y2.dat @data/y2.dat

httpie -a olenagerasimova:*** PUT https://central.artipie.com/olenagerasimova/data/y3.dat @data/y3.dat

As this binary repository API is very simple (HTTP PUT and GET requests), it's easy to write a piece of code in any language to upload and download the required files.

Python project

The source code of an example Python project is available from my GitHub repository. The main idea of the example is to download three data files from Artipie Central, read the numbers into arrays, and use these arrays to draw a plot. Use pip to install the example package and run it:

​$ python3 -m pip install --index-url \
https://central.artipie.com/olenagerasimova/pypi/ \
pypiexample
$ python3 -m pypiexample
​

By setting the --index-url to the Artipie Central Python repository, pip downloads the packages from it rather than the PyPi repository that serves as the usual default. After running the commands, a polar plot with three curves, a visualization of the data files is displayed.

To publish the package to the Artipie Central repository, build it with and use twine to upload it:

commandline
$ python setup.py sdist bdist_wheel

$ twine upload --repository-url \
https://central.artipie.com/olenagerasimova/pypi
-u olenagerasimova -p *** dist/*

That's how easy it is to set up a files repositories in Artipie Central, create a sample Python project, publish, and install it. You don't have to use Artipie Central, though. Artipie can be self-hosted, so you can run a repository on your own local network.

Run Artipie as a container

Running Artipie as a container makes setup as easy as installing either Podman or Docker. Assuming you have one of these installed, open a terminal:

​$ podman run -it -p 8080:8080 -p 8086:8086 artipie/artipie:latest
​

This starts a new container running the latest Artipie version. It also maps two ports. Your repositories are served on port 8080. The Artipie Rest API and Swagger documentation are provided on port 8086. A new image generates a default configuration, printing a list of running repositories, test credentials, and a link to the Swagger documentation to your console.

You can also use the Artipie Rest API to see existing repositories:

1. Go to the Swagger documentation page at http://localhost:8086/api/index-org.html.

2. In the Select a definition list, choose Auth token

3. Generate and copy the authentication token for the user artipie with the password artipie

4. Switch to the Repositories definition and click the Authorize button, and then paste in the token

Image by:

(Seth Kenlon, CC BY-SA 4.0)

Perform a GET request for /api/v1/repository/list. In response, you receive a JSON list with three default repositories:

​[ "artipie/my-bin",
  "artipie/my-docker",
  "artipie/my-maven" ]

The Python repository isn't included in the default configuration. You can correct that by performing a PUT request to /api/v1/repository/{user}/{repo} from the  Swagger interface. In this case, user is the name of the default user (artipie) and repo is the name of the new repository. You can call your new Python repository my-pypi. Here's an example request body, containing a JSON object with the repository settings:

​{ "repo": {
  "type": "pypi",
  "storage": "default",
  "permissions": {
  "*": [
  "download"
  ],
  "artipie": [
  "upload"
  ] }
 } }

All the JSON fields are the same as when you create a repository in the dashboard in YAML format. The type of our repository is pypi, the default storage is used, and anyone can download but only the user artipie can upload.

Make a GET request to /api/v1/repository/list again to make sure your repository was created. Now, you have four repositories:

​[ "artipie/my-bin",
"artipie/my-docker",
"artipie/my-maven",
"artipie/my-pypi" ]

You've created your own Artipie installation, containing several repositories! The Artipie image can run both on a personal computer or on a remote server inside a private network. You can use it to exchange packages within a company, group, or university. It's an easy way to set up your own software services, and it's not just for Python. Take some time to explore Artipie and see what it can make possible for you.