Chapter 3. File download support

Chapter 3. File download support
Prev		Next

Table of Contents

Overview
Local file fetcher
CVS fetcher
HTTP/FTP fetcher
SVN fetcher
GIT fetcher

Overview

BitBake provides support to download files this procedure is called fetching and it handled by the fetch and fetch2 modules. At this point the original fetch code is considered to be replaced by fetch2 and this manual only related to the fetch2 codebase.

The SRC_URI is normally used to tell BitBake which files to fetch. The next sections will describe the available fetchers and their options. Each fetcher honors a set of variables and per URI parameters separated by a “;” consisting of a key and a value. The semantics of the variables and parameters are defined by the fetcher. BitBake tries to have consistent semantics between the different fetchers.

The overall fetch process is that first, fetches are attempted from PREMIRRORS. If those don't work, the original SRC_URI is attempted and if that fails, BitBake will fall back to MIRRORS. Cross urls are supported, so its possible to mirror a git repository on an http server as a tarball for example. Some example commonly used mirror definitions are:

PREMIRRORS ?= "\
bzr://.*/.*   http://somemirror.org/sources/ \n \
cvs://.*/.*   http://somemirror.org/sources/ \n \
git://.*/.*   http://somemirror.org/sources/ \n \
hg://.*/.*    http://somemirror.org/sources/ \n \
osc://.*/.*   http://somemirror.org/sources/ \n \
p4://.*/.*    http://somemirror.org/sources/ \n \
svk://.*/.*   http://somemirror.org/sources/ \n \
svn://.*/.*   http://somemirror.org/sources/ \n"

MIRRORS =+ "\
ftp://.*/.*      http://somemirror.org/sources/ \n \
http://.*/.*     http://somemirror.org/sources/ \n \
https://.*/.*    http://somemirror.org/sources/ \n"

Non-local downloaded output is placed into the directory specified by the DL_DIR. For non local archive downloads the code can verify sha256 and md5 checksums for the download to ensure the file has been downloaded correctly. These may be specified either in the form SRC_URI[md5sum] for the md5 checksum and SRC_URI[sha256sum] for the sha256 checksum or as parameters on the SRC_URI such as SRC_URI="http://example.com/foobar.tar.bz2;md5sum=4a8e0f237e961fd7785d19d07fdb994d". If BB_STRICT_CHECKSUM is set, any download without a checksum will trigger an error message. In cases where multiple files are listed in SRC_URI, the name parameter is used assign names to the urls and these are then specified in the checksums in the form SRC_URI[name.sha256sum].