Great, now you have a working knowledge of programming in Julia. When you use it to make awesome packages, you’ll no doubt want to publish them. So, today we’re going to be giving you an overview of packaging in Julia.
Before we start, there are some terms that you should be familiar with. For the sake of completeness, we’ll discuss them briefly, feel free to skip this section. When you code in Julia, you work in projects, which are composed of files with the
.jl extension as well as a manifest. Artefacts are platform-specific binaries (that aren’t in Julia). At the risk of oversimplification, they can be thought of as the
.bin files that tag along with everything nice.
When you bundle a project with all its dependencies, you get a package. And as we already know, packages themselves are the building blocks that supplement the standard library to add niche-specific functionality. And environments then, are defined by a list of packages and their respective versions. This is stored in a manifest stored in
Lights Camera Code
I mean action, lights camera action!… I mean lets look at all the things we talked about, but in actuality. Lets create a new environment:
(@v1.5) pkg> activate foo Activating new environment at `C:\Users\MLG\Desktop\foo\Project.toml`
(foo) pkg> status Status `C:\Users\MLG\Desktop\foo\Project.toml` (empty project)
Now, if you were to try to go to that directory, you’d find that it doesn’t even exist! That’s because a blank Project might as well be non-existent, so it is!! So lets add a package:
(foo) pkg> add Pluto Resolving package versions... Updating `C:\Users\MLG\Desktop\foo\Project.toml` [c3e4b0f8] + Pluto v0.12.17 Updating `C:\Users\MLG\Desktop\foo\Manifest.toml` [56f22d72] + Artifacts v1.3.0 [9a962f9c] + DataAPI v1.4.0 [e2d170a0] + DataValueInterfaces v1.0.0 [fb4132e2] + FuzzyCompletions v0.4.0 [cd3eb016] + HTTP v0.9.0 [83e8ac13] + IniFile v0.5.0  + IteratorInterfaceExtensions v1.0.0 [692b3bcd] + JLLWrappers v1.1.3 [739be429] + MbedTLS v1.0.3 [c8ffd9c3] + MbedTLS_jll v2.16.8+1 [99f44e22] + MsgPack v1.1.0 [c3e4b0f8] + Pluto v0.12.17 [3783bdb8] + TableTraits v1.0.0 [bd369af6] + Tables v1.2.2 [5c2747f8] + URIs v1.1.0 [2a0f44e3] + Base64 [ade2ca70] + Dates [8ba89e20] + Distributed [b77e0a4c] + InteractiveUtils [76f85450] + LibGit2 [8f399da3] + Libdl [37e2e46d] + LinearAlgebra [56ddb016] + Logging [d6f4376e] + Markdown [44cfe95a] + Pkg [de0858da] + Printf [3fa0cd96] + REPL [9a3f8284] + Random [ea8e919c] + SHA [9e88b42a] + Serialization [6462fe0b] + Sockets [8dfed614] + Test [cf7118a7] + UUIDs [4ec0a83e] + Unicode
Last time we glossed over this, but not this time. So, you see here, not only did
Pluto(its latest version at that) but also installed all it’s dependencies. So, now if you look at your
Manifest.toml you’ll see it to be thus:
# This file is machine-generated - editing it directly is not advised [[Artifacts]] deps = ["Pkg"] git-tree-sha1 = "c30985d8821e0cd73870b17b0ed0ce6dc44cb744" uuid = "56f22d72-fd6d-98f1-02f0-08ddc0907c33" version = "1.3.0" [[Base64]] uuid = "2a0f44e3-6c83-55bd-87e4-b1978d98bd5f" [[DataAPI]] git-tree-sha1 = "ad84f52c0b8f05aa20839484dbaf01690b41ff84" uuid = "9a962f9c-6df0-11e9-0e5d-c546b8b5ee8a" version = "1.4.0" [[DataValueInterfaces]] git-tree-sha1 = "bfc1187b79289637fa0ef6d4436ebdfe6905cbd6" uuid = "e2d170a0-9d28-54be-80f0-106bbe20a464" version = "1.0.0" [[Dates]] deps = ["Printf"] uuid = "ade2ca70-3891-5945-98fb-dc099432e06a" [[Distributed]] deps = ["Random", "Serialization", "Sockets"] uuid = "8ba89e20-285c-5b6f-9357-94700520ee1b" [[FuzzyCompletions]] deps = ["REPL"] git-tree-sha1 = "5ca3ddf3061771d25d1699ce53a80a39300811e3" uuid = "fb4132e2-a121-4a70-b8a1-d5b831dcdcc2" version = "0.4.0" [[HTTP]] deps = ["Base64", "Dates", "IniFile", "MbedTLS", "Sockets", "URIs"] git-tree-sha1 = "9634200f8e16554cb1620dfb20501483b873df86" uuid = "cd3eb016-35fb-5094-929b-558a96fad6f3" version = "0.9.0" [[IniFile]] deps = ["Test"] git-tree-sha1 = "098e4d2c533924c921f9f9847274f2ad89e018b8" uuid = "83e8ac13-25f8-5344-8a64-a9f2b223428f" version = "0.5.0" [[InteractiveUtils]] deps = ["Markdown"] uuid = "b77e0a4c-d291-57a0-90e8-8db25a27a240" [[IteratorInterfaceExtensions]] git-tree-sha1 = "a3f24677c21f5bbe9d2a714f95dcd58337fb2856" uuid = "82899510-4779-5014-852e-03e436cf321d" version = "1.0.0" [[JLLWrappers]] git-tree-sha1 = "c70593677bbf2c3ccab4f7500d0f4dacfff7b75c" uuid = "692b3bcd-3c85-4b1f-b108-f13ce0eb3210" version = "1.1.3" [[LibGit2]] deps = ["Printf"] uuid = "76f85450-5226-5b5a-8eaa-529ad045b433" [[Libdl]] uuid = "8f399da3-3557-5675-b5ff-fb832c97cbdb" [[LinearAlgebra]] deps = ["Libdl"] uuid = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" [[Logging]] uuid = "56ddb016-857b-54e1-b83d-db4d58db5568" [[Markdown]] deps = ["Base64"] uuid = "d6f4376e-aef5-505a-96c1-9c027394607a" [[MbedTLS]] deps = ["Dates", "MbedTLS_jll", "Random", "Sockets"] git-tree-sha1 = "1c38e51c3d08ef2278062ebceade0e46cefc96fe" uuid = "739be429-bea8-5141-9913-cc70e7f3736d" version = "1.0.3" [[MbedTLS_jll]] deps = ["Artifacts", "JLLWrappers", "Libdl", "Pkg"] git-tree-sha1 = "0eef589dd1c26a3ac9d753fe1a8bcad63f956fa6" uuid = "c8ffd9c3-330d-5841-b78e-0817d7145fa1" version = "2.16.8+1" [[MsgPack]] deps = ["Serialization"] git-tree-sha1 = "a8cbf066b54d793b9a48c5daa5d586cf2b5bd43d" uuid = "99f44e22-a591-53d1-9472-aa23ef4bd671" version = "1.1.0" [[Pkg]] deps = ["Dates", "LibGit2", "Libdl", "Logging", "Markdown", "Printf", "REPL", "Random", "SHA", "UUIDs"] uuid = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" [[Pluto]] deps = ["Base64", "Dates", "Distributed", "FuzzyCompletions", "HTTP", "InteractiveUtils", "Logging", "Markdown", "MsgPack", "Pkg", "REPL", "Sockets", "Tables", "UUIDs"] git-tree-sha1 = "41927b687ee3c553dd8dac28f4183326b054bd54" uuid = "c3e4b0f8-55cb-11ea-2926-15256bba5781" version = "0.12.17" [[Printf]] deps = ["Unicode"] uuid = "de0858da-6303-5e67-8744-51eddeeeb8d7" [[REPL]] deps = ["InteractiveUtils", "Markdown", "Sockets"] uuid = "3fa0cd96-eef1-5676-8a61-b3b8758bbffb" [[Random]] deps = ["Serialization"] uuid = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c" [[SHA]] uuid = "ea8e919c-243c-51af-8825-aaa63cd721ce" [[Serialization]] uuid = "9e88b42a-f829-5b0c-bbe9-9e923198166b" [[Sockets]] uuid = "6462fe0b-24de-5631-8697-dd941f90decc" [[TableTraits]] deps = ["IteratorInterfaceExtensions"] git-tree-sha1 = "b1ad568ba658d8cbb3b892ed5380a6f3e781a81e" uuid = "3783bdb8-4a98-5b6b-af9a-565f29a5fe9c" version = "1.0.0" [[Tables]] deps = ["DataAPI", "DataValueInterfaces", "IteratorInterfaceExtensions", "LinearAlgebra", "TableTraits", "Test"] git-tree-sha1 = "240d19b8762006ff04b967bdd833269ad642d550" uuid = "bd369af6-aec1-5ad0-b16a-f7cc5008161c" version = "1.2.2" [[Test]] deps = ["Distributed", "InteractiveUtils", "Logging", "Random"] uuid = "8dfed614-e22c-5e08-85e1-65c5234f0b40" [[URIs]] git-tree-sha1 = "bc331715463c41d601cf8bfd38ca70a490af5c5b" uuid = "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4" version = "1.1.0" [[UUIDs]] deps = ["Random", "SHA"] uuid = "cf7118a7-6976-5b1a-9a39-7adc72f591a4" [[Unicode]] uuid = "4ec0a83e-493e-50e2-b9ac-8f72acf5a8f5"
Long and tedious indeed. But this is not the manifest we’ve been talking about. No, the manifest is in the same directory,
[deps] Pluto = "c3e4b0f8-55cb-11ea-2926-15256bba5781"
Now this looks more readable. And this is all that your project is currently. But now you understand that whenever you clone a WIP project, which isn’t yet packaged, it’ll come with its own
Project.toml which Julia will use to reproducibly build that project on your machine. To accomplish this, simply go into the project directory and:
(foo) pkg> activate .
Now that you know all about projects, lets move on to packages.
A package is really just a clothed project. More precisely,
A package is a project with a name,
uuidand version entry in the
Project.tomlfile, and a
src/PackageName.jlfile that defines the module
PackageName. This file is executed when the package is loaded.
Without further ado lets create one and see for ourselves:
(fooProj) pkg> generate barPackage Generating project barPackage: barPackage\Project.toml barPackage\src/barPackage.jl
So we see that we’ve got a new directory and some files. In particular, we have a new
Project.toml while our original
Project.toml are unchanged. Indeed, we’ve just created a package
barPackage in our project
fooProject. So lets import our project’s package into it:
julia> cd("barPackage") (fooProj) pkg> activate . Activating environment at `C:\Users\icebear\Desktop\fooProj\barPackage\Project.toml` julia> import barPackage [ Info: Precompiling barPackage [317e392e-7f8c-4046-94eb-a62d137665d3] julia> barPackage.greet() Hello World!
And we even tested it out, without having to open any files, great. Now you can go on and publish this anywhere.
When you install Julia, you have only one project, named after the Julia version you installed, located at
Although all the packages themselves are stored together (in
.julia/packages directory), so that you avoid re-downloading package versions that you already have.
A novel thing about environments in Julia is that they’re stackable, as in you could be working in an environment
foo, having some packages, and then overlay an environment
bar on top of it to have packages from both the environments. Just a reminder, environments are independently updated, so updating
foo won’t change
bar at all.
Finally, something to note is that
Pkg has native support for multiple registries. We won’t go deeper into this, but in summary it means that you could, say be working with a public package, fork it into a private registry and then supply your peers with that forked version without having to jump through unnecessary hoops. In fact, forking a package is as simple as:
(foo) pkg> develop --local Pluto
This will setup a git clone of the
Pluto package in your working directory, and then make the environment
foo use that local version as opposed to the publicly available one. But remember that this stops the updating of this package’s dependency graph in the manifest, you have to trigger it manually hereon, by using
resolve in the REPL. If you want to go back to the publicly available of a package you earlier cloned using
develop, you’ll use:
(foo) pkg> free Pluto
As expected, you can import a package only once. If you want to make changes to (even a local) package, and then load the updated version, use Revise.jl. At the opposite end, if you want a package to stay untouched, and not be updated at all:
(foo) pkg> pin Pluto
This of course pertains only to that environment.
If a package or artifact is not used by any Project, then julia labels it an orphan. Orphans that are more than a month old are deleted whenever Pkg garbage collects. You can also trigger it manually:
(foo) pkg> gc
This will delete all orphans present at that time.