All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
main.config.json version we generate is no
longer hard-coded to 1.0.0. We now use the version of the spec we have
imported (with any -dev suffix stripped, as such a prefix causes havoc with
verification tools – ideally we would only ever use released versions of the
spec but that’s not always possible). #452cgroup namespace to the default configuration generated by umoci unpack to make sure that our configuration plays nicely with runc when on
cgroupv2 systems.VerifiedReadCloser hardening work (to read all trailing bytes) which would
cause walk operations on images to hash every blob in the image (even blobs
which we couldn’t parse and thus couldn’t recurse into). To resolve this, we
no longer recurse into unparseable blobs. #373 #375 #394EINTR on io.Copy operations. Newer Go versions have added more
opportunistic pre-emption which can cause EINTR errors in io paths that
didn’t occur before. #437umoci unpack or
umoci raw unpack) that contained a symlink entry for /., umoci would
apply subsequent layers to the target of the symlink (resolved on the host
filesystem). This means that if you ran umoci as root, a malicious image
could overwrite any file on the system (assuming you didn’t have any other
access control restrictions). CVE-2021-29136umoci has been adopted by the Open Container Initative as a reference implementation of the OCI Image Specification. This will have little impact on the roadmap or scope of umoci, but it does further solidify umoci as a useful piece of “boring container infrastructure” that can be used to build larger systems.
As part of the adoption procedure, the import path and module name of umoci
has changed from github.com/openSUSE/umoci to
github.com/opencontainers/umoci. This means that users of our (still
unstable) Go API will have to change their import paths in order to update to
newer versions of umoci.
The old GitHub project will contain a snapshot of v0.4.5 with a few minor
changes to the readme that explain the situation. Go projects which import
the archived project will receive build warnings that explain the need to
update their import paths.
umoci config, which
often takes --prefixed flag arguments. opencontainers/umoci#328type: bind for generated config.json bind-mounts. While this doesn’t
make too much sense (see opencontainers/runc#2035), it does mean that
rootless containers work properly with newer runc releases (which appear to
have regressed when handling file-based bind-mounts with a “bad” type).
opencontainers/umoci#294 opencontainers/umoci#295--history.* options can now decide to omit a
history entry with --no-history. Note that while this is supported for
commands that create layers (umoci repack, umoci insert, and umoci raw add-layer) it is not recommended to use it for those commands since it can
cause other tools to become confused when inspecting the image history. The
primary usecase is to allow umoci config --no-history to leave no traces in
the history. See SUSE/kiwi#871. opencontainers/umoci#270umoci insert now has a --tag option that allows you to non-destructively
insert files into an image. The semantics match umoci config --tag.
opencontainers/umoci#273umoci unpack --keep-dirlinks (in the same vein as rsync’s flag with
the same name) which allows layers that contain entries which have a symlink
as a path component. opencontainers/umoci#246umoci insert now supports whiteouts in two significant ways. You can use
--whiteout to “insert” a deletion of a given path, while you can use
--opaque to replace a directory by adding an opaque whiteout (the default
behaviour causes the old and new directories to be merged).
opencontainers/umoci#257umoci unpack now no longer erases system.nfs4_acl and also has some more
sophisticated handling of forbidden xattrs. opencontainers/umoci#252
opencontainers/umoci#248umoci unpack now appears to work correctly on SELinux-enabled systems
(previously we had various issues where umoci wouldn’t like it when it was
trying to ensure the filesystem was reproducibly generated and SELinux xattrs
would act strangely). To fix this, now umoci unpack will only cause errors
if it has been asked to change a forbidden xattr to a value different than
it’s current on-disk value. opencontainers/umoci#235 opencontainers/umoci#259umoci subcommands has
increased significantly due to an expansion in the specification of the
format of the ref.name annotation. To quote the specification, the
following is the EBNF of valid refname values. opencontainers/umoci#234
refname ::= component ("/" component)*
component ::= alphanum (separator alphanum)*
alphanum ::= [A-Za-z0-9]+
separator ::= [-._:@+] | "--"
umoci insert subcommand which adds a given file to a path inside the
container. opencontainers/umoci#237umoci raw unpack subcommand in order to allow users to unpack images
without needing a configuration or any of the manifest generation.
opencontainers/umoci#239umoci how has a logo. Thanks to Max Bailey for contributing
this to the project. opencontainers/umoci#165 opencontainers/umoci#249umoci unpack now handles out-of-order regular whiteouts correctly (though
this ordering is not recommended by the spec – nor is it required). This is
an extension of opencontainers/umoci#229 that was missed during review.
opencontainers/umoci#232umoci unpack and umoci repack now make use of a far more optimised gzip
compression library. In some benchmarks this has resulted in umoci repack
speedups of up to 3x (though of course, you should do your own benchmarks).
umoci unpack unfortunately doesn’t have as significant of a performance
improvement, due to the nature of gzip decompression (in future we may
switch to zlib wrappers). opencontainers/umoci#225 opencontainers/umoci#233umoci repack now supports --refresh-bundle which will update the
OCI bundle’s metadata (mtree and umoci-specific manifests) after packing the
image tag. This means that the bundle can be used as a base layer for
future diffs without needing to unpack the image again. opencontainers/umoci#196umo.ci. opencontainers/umoci#188user.rootlesscontainers specification, which allows
for persistent on-disk emulation of chown(2) inside rootless containers.
This implementation is interoperable with @AkihiroSuda’s PRoot
fork (though we do not test its interoperability at the
moment) as both tools use the same protobuf
specification. opencontainers/umoci#227umoci unpack now has support for opaque whiteouts (whiteouts which remove
all children of a directory in the lower layer), though umoci repack does
not currently have support for generating them. While this is technically a
spec requirement, through testing we’ve never encountered an actual user of
these whiteouts. opencontainers/umoci#224 opencontainers/umoci#229umoci unpack will now use some rootless tricks inside user namespaces for
operations that are known to fail (such as mknod(2)) while other operations
will be carried out as normal (such as lchown(2)). It should be noted that
the /proc/self/uid_map checking we do can be tricked into not detecting
user namespaces, but you would need to be trying to break it on purpose.
opencontainers/umoci#171 opencontainers/umoci#230umoci unpack will now “clean up” the bundle generated if an error occurs
during unpacking. Previously this didn’t happen, which made cleaning up the
responsibility of the caller (which was quite difficult if you were
unprivileged). This is a breaking change, but is in the error path so it’s
not critical. opencontainers/umoci#174 opencontainers/umoci#187umoci gc now will no longer remove unknown files and directories that
aren’t flock(2)ed, thus ensuring that any possible OCI image-spec
extensions or other users of an image being operated on will no longer
break. opencontainers/umoci#198umoci unpack --rootless will now correctly handle regular file unpacking
when overwriting a file that umoci doesn’t have write access to. In
addition, the semantics of pre-existing hardlinks to a clobbered file are
clarified (the hard-links will not refer to the new layer’s inode).
opencontainers/umoci#222 opencontainers/umoci#223hack/release.sh that caused the release artefacts
to not match the intended style, as well as making it more generic so other
projects can use it. opencontainers/umoci#155 opencontainers/umoci#163go vet and go lint to not run as part
of our CI jobs. This means that some of the information submitted as part of
CII best practices badging was not accurate. This has been corrected,
and after review we concluded that only stylistic issues were discovered by
static analysis. opencontainers/umoci#158umoci unpack would not correctly preserve set{uid,gid} bits. While this
would not cause issues when building an image (as we only create a manifest
of the final extracted rootfs), it would cause issues for other users of
umoci. opencontainers/umoci#166 opencontainers/umoci#169go-mtree, which fixes several minor
bugs with manifest generation. opencontainers/umoci#176umoci unpack would not handle “weird” tar archive layers previously (it
would error out with DiffID errors). While this wouldn’t cause issues for
layers generated using Go’s archive/tar implementation, it would cause
issues for GNU gzip and other such tools. opencontainers/umoci#178
opencontainers/umoci#179umoci unpack’s mapping options (--uid-map and --gid-map) have had an
interface change, to better match the user_namespaces(7)
interfaces. Note that this is a breaking change, but the workaround is to
switch to the trivially different (but now more consistent) format.
opencontainers/umoci#167umoci unpack used to create the bundle and rootfs with world
read-and-execute permissions by default. This could potentially result in an
unsafe rootfs (containing dangerous setuid binaries for instance) being
accessible by an unprivileged user. This has been fixed by always setting the
mode of the bundle to 0700, which requires a user to explicitly work around
this basic protection. This scenario was documented in our security
documentation previously, but has now been fixed. opencontainers/umoci#181
opencontainers/umoci#182umoci now passes all of the requirements for the CII best practices bading
program. opencontainers/umoci#134umoci also now has more extensive architecture, quick-start and roadmap
documentation. opencontainers/umoci#134umoci now supports 1.0.0 of the OCI image
specification and 1.0.0 of the OCI runtime
specification, which are the first milestone release. Note
that there are still some remaining UX issues with --image and other parts
of umoci which may be subject to change in future versions. In particular,
this update of the specification now means that images may have ambiguous
tags. umoci will warn you if an operation may have an ambiguous result, but
we plan to improve this functionality far more in the future.
opencontainers/umoci#133 opencontainers/umoci#142umoci also now supports more complicated descriptor walk structures, and
also handles mutation of such structures more sanely. At the moment, this
functionality has not been used “in the wild” and umoci doesn’t have the UX
to create such structures (yet) but these will be implemented in future
versions. opencontainers/umoci#145umoci repack now supports --mask-path to ignore changes in the rootfs
that are in a child of at least one of the provided masks when generating new
layers. opencontainers/umoci#127github.com/opencontainers/umoci/oci/cas/drivers/dir actually
make sense now. opencontainers/umoci#121umoci unpack now generates config.json blobs according to the still
proposed OCI image specification conversion document.
opencontainers/umoci#120umoci repack also now automatically adding Config.Volumes from the image
configuration to the set of masked paths. This matches recently added
recommendations by the spec, but is a backwards-incompatible
change because the new default is that Config.Volumes will be masked.
If you wish to retain the old semantics, use --no-mask-volumes (though make
sure to be aware of the reasoning behind Config.Volume masking).
opencontainers/umoci#127umoci now uses SecureJoin rather than a patched version of
FollowSymlinkInScope. The two implementations are roughly equivalent, but
SecureJoin has a nicer API and is maintained as a separate project.golang.org/x/sys/unix over syscall where possible,
which makes the codebase significantly cleaner. opencontainers/umoci#141hack/release.sh automates the process of generating all of the published
artefacts for releases. The new script also generates signed source code
archives. opencontainers/umoci#116umoci now outputs configurations that are compliant with v1.0.0-rc5 of
the OCI runtime-spec. This means that now you can use runc
v1.0.0-rc3 with umoci (and rootless containers should work out of the box
if you use a development build of runc). opencontainers/umoci#114umoci unpack no longer adds a dummy linux.seccomp entry, and instead just
sets it to null. opencontainers/umoci#114umoci now has some automated scripts for generated RPMs that are used in
openSUSE to automatically submit packages to OBS. opencontainers/umoci#101--clear=config.{cmd,entrypoint} is now supported. While this interface is a
bit weird (cmd and entrypoint aren’t treated atomically) this makes the
UX more consistent while we come up with a better cmd and entrypoint UX.
opencontainers/umoci#107umoci raw runtime-config. It generates the runtime-spec
config.json for a particular image without also unpacking the root
filesystem, allowing for users of umoci that are regularly parsing
config.json without caring about the root filesystem to be more efficient.
However, a downside of this approach is that some image-spec fields
(Config.User) require a root filesystem in order to make sense, which is
why this command is hidden under the umoci-raw(1) subcommand (to make sure
only users that understand what they’re doing use it). opencontainers/umoci#110umoci’s oci/cas and oci/config libraries have been massively refactored
and rewritten, to allow for third-parties to use the OCI libraries. The plan
is for these to eventually become part of an OCI project. opencontainers/umoci#90oci/cas interface has been modifed to switch from *ispec.Descriptor
to ispec.Descriptor. This is a breaking, but fairly insignificant, change.
opencontainers/umoci#89umoci now uses an updated version of go-mtree, which has a complete
rewrite of Vis and Unvis. The rewrite ensures that unicode handling is
handled in a far more consistent and sane way. opencontainers/umoci#88umoci used to set process.user.additionalGids to the “normal value” when
unpacking an image in rootless mode, causing issues when trying to actually
run said bundle with runC. opencontainers/umoci#109CHANGELOG.md has now been added. opencontainers/umoci#76umoci now supports v1.0.0-rc4 images, which has made fairly minimal
changes to the schema (mainly related to mediaTypes). While this change
is backwards compatible (several fields were removed from the schema, but
the specification allows for “additional fields”), tools using older versions
of the specification may fail to operate on newer OCI images. There was no UX
change associated with this update.umoci tag would fail to clobber existing tags, which was in contrast to how
the rest of the tag clobbering commands operated. This has been fixed and is
now consistent with the other commands. opencontainers/umoci#78umoci repack now can correctly handle unicode-encoded filenames, allowing
the creation of containers that have oddly named files. This required fixes
to go-mtree (where the issue was). opencontainers/umoci#80%check of an rpmbuild script, allowing
for proper testing. opencontainers/umoci#65.unpack, repack: xattr support which also handles security.selinux.*
difficulties. opencontainers/umoci#49 opencontainers/umoci#52config, unpack: Ensure that environment variables are not duplicated in
the extracted or stored configurations. opencontainers/umoci#30--rootless if umoci fails with EPERM.--debug flag was given to umoci.
This requires a patch to pkg/errors.gc: Garbage collection now also garbage collects temporary directories.
opencontainers/umoci#17go-mtree so that it’s much more
upstream-friendly.unpack, repack: Support for rootless unpacking and repacking.
opencontainers/umoci#26unpack, repack: UID and GID mapping when unpacking and repacking.
opencontainers/umoci#26tag, rm, ls: Tag modification commands such as umoci tag, umoci rm
and umoci ls. opencontainers/umoci#6 opencontainers/umoci#27stat: Output information about an image. Currently only shows the history
information. Only the JSON output is stable. opencontainers/umoci#38init, new: New commands have been created to allow for image creation
from scratch. opencontainers/umoci#5 opencontainers/umoci#42gc: Garbage collection of images. opencontainers/umoci#6unpack, repack: Create history entries automatically (with options to
modify the entries). opencontainers/umoci#36unpack: Store information about its source to ensure consistency when doing
a repack. opencontainers/umoci#14--image and --from arguments have been combined into a single
<path>[:<tag>] argument for --image. opencontainers/umoci#39unpack: Configuration annotations are now extracted, though there are still
some discussions happening upstream about the correct way of doing this.
opencontainers/umoci#43repack: Errors encountered during generation of delta layers are now
correctly propagated. opencontainers/umoci#33unpack: Hardlinks are now extracted as real hardlinks. opencontainers/umoci#25unpack, repack: Symlinks are now correctly resolved inside the unpacked
rootfs. opencontainers/umoci#27unpackrepackconfig