| blob | 925b3e2cdc27709a987fe305b586ca14e69c4aeb |
1 .\" $NetBSD: pkg_comp.8,v 1.36 2012/02/27 22:42:27 jmmv Exp $
2 .\"
3 .\" pkg_comp - Build packages inside a clean chroot environment
4 .\" Copyright (c) 2002, 2003, 2004, 2005 Julio M. Merino Vidal <jmmv@NetBSD.org>
5 .\"
6 .\" Redistribution and use in source and binary forms, with or without
7 .\" modification, are permitted provided that the following conditions
8 .\" are met:
9 .\" 1. Redistributions of source code must retain the above copyright
10 .\" notice, this list of conditions and the following disclaimer.
11 .\" 2. Neither the name of The NetBSD Foundation nor the names of its
12 .\" contributors may be used to endorse or promote products derived
13 .\" from this software without specific prior written permission.
14 .\" 3. Neither the name of author nor the names of its contributors may
15 .\" be used to endorse or promote products derived from this software
16 .\" without specific prior written permission.
17 .\"
18 .\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
19 .\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
20 .\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
21 .\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
22 .\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
23 .\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
24 .\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
25 .\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
26 .\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
27 .\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
28 .\" POSSIBILITY OF SUCH DAMAGE.
29 .\"
30 .Dd May 11, 2012
31 .Dt PKG_COMP 8
32 .Os
33 .Sh NAME
34 .Nm pkg_comp
35 .Nd build packages inside a sandbox
36 .Sh SYNOPSIS
37 .Nm
38 .Oo Fl Po
39 .Cm c Ns \&| Ns Cm C
40 .Pc
41 .Ar conf_file
42 .Oc
43 .Op Fl Nn
44 .Ar target
45 .Op Ar pkg_name ...
46 .Sh DESCRIPTION
47 .Nm ,
48 or
49 .Em Package Compiler
50 in its full name,
51 is a tool that makes easy the compilation of packages inside a clean
52 sandbox.
53 This allows an easy tracking of exact dependencies
54 and the correct behavior of a package in a fresh system installation.
55 .Pp
56 The behavior of
57 .Nm
58 is controlled through a small configuration file and a target (keep
59 reading to learn more).
60 The configuration file tells
61 .Nm
62 how to configure the new chroot environment, and the target specifies
63 which action to take.
64 .Pp
65 The following options are recognized:
66 .Bl -tag -width XcXconf_file
67 .It Fl C Ar conf_file
68 Use
69 .Ar conf_file
70 as configuration file (full path expected).
71 .It Fl c Ar conf_file
72 Use
73 .Ar conf_file
74 as configuration file (only base name expected).
75 .It Fl N
76 With the exception of
77 .Pa pkgtools/libkver
78 (see
79 .Va NETBSD_RELEASE )
80 avoid installation of default packages as well as
81 .Va INSTALL_PACKAGES
82 and
83 .Va BUILD_PACKAGES
84 during the creation of the chroot.
85 .It Fl n
86 Avoid installation of
87 .Va INSTALL_PACKAGES
88 and
89 .Va BUILD_PACKAGES
90 during the creation of the chroot.
91 .El
92 .Ss What to use it for?
93 You can use
94 .Nm
95 to achieve many goals when building packages.
96 Here are some ideas:
97 .Bl -bullet
98 .It
99 Build packages for other system versions.
100 For example, build packages for
101 .Nx 4.0
102 while you are running
103 .Nx 5.0 .
104 .It
105 Build packages using different
106 .Pa mk.conf
107 options than your current system, like changing the threading library,
108 .Sy COPTS ,
109 placement of configuration files, etc.
110 .It
111 Debug the build process of a package, checking if buildlinks work
112 properly.
113 .It
114 Avoid autoconf's side effects by keeping a separate chroot for each
115 project, like one for GNOME2 and another one for KDE3.
116 .It
117 Schedule builds of package sets for several different machines.
118 .El
119 .Sh CONTROL DIRECTORY
120 .Nm
121 needs to store several pieces of information when it is running.
122 Instead of using normal system trees, it uses a special directory inside the
123 chroot to avoid polluting the system.
124 It stores there scripts, object files, built packages, etc.
125 This directory is
126 .Pa $DESTDIR/pkg_comp ;
127 the symbolic link
128 .Pa $DESTDIR/p
129 is automatically created to ease pathnames when working inside the chroot.
130 .Sh CONFIGURATION
131 With
132 .Nm
133 you can maintain several configuration files so you can work with
134 different chroot jails easily.
135 To make this easy, configuration files are stored inside
136 .Pa $HOME/pkg_comp ,
137 followed by the configuration file name and the .conf suffix.
138 The default configuration file is
139 .Pa $HOME/pkg_comp/default.conf ,
140 and is always used if you do not specify another one.
141 The configuration file name is specified by the argument of the
142 .Fl c
143 option.
144 Alternatively you can specify any pathname as a configuration file
145 with the argument of the
146 .Fl C
147 option.
148 .Pp
149 Configuration files are simple shell scripts that define
150 variables.
151 The default values shown here are those written in the template when
152 issuing a maketemplate.
153 .Bl -tag -width indent
154 .It AUTO_PACKAGES
155 A list of packages to automatically build during the
156 .Sy auto
157 target.
158 A package is in the form
159 .Sq section/name ,
160 like
161 .Sq misc/colorls ,
162 or a plain name like
163 .Sq colorls .
164 Defaults to nothing.
165 .It AUTO_TARGET
166 The pkgsrc target to use when building packages in an automated fashion
167 (using the
168 .Ql auto
169 target).
170 Should be set to
171 .Ql package
172 or
173 .Ql bin-install ,
174 as other values are useless.
175 Defaults to
176 .Ql package .
177 .It BUILD_PACKAGES
178 A list of packages to automatically build after the
179 .Sy makeroot
180 target.
181 A package is in the form
182 .Sq section/name ,
183 like
184 .Sq misc/colorls ,
185 or a plain name like
186 .Sq colorls .
187 Defaults to nothing.
188 .It BUILD_TARGET
189 The pkgsrc target to use when building packages.
190 It can contain any target supported by the pkgsrc system, but
191 reasonable values are:
192 .Ql install ,
193 .Ql package
194 and
195 .Ql bin-install .
196 Defaults to
197 .Ql package .
198 .It COPYROOTCFG
199 If set to
200 .Ql yes ,
201 all configuration files (not directories) that reside inside
202 .Pa /root
203 are copied to
204 .Sy $DESTDIR/root .
205 Defaults to
206 .Ql no .
207 .It DESTDIR
208 The chroot jail directory.
209 Defaults to
210 .Pa /var/chroot/pkg_comp/default .
211 .It DISTRIBDIR
212 This is the directory which holds
213 .Nb
214 binary sets and X sets.
215 Its structure is the same as official release
216 distributions, that is, tgz files must reside inside
217 .Pa $DISTRIBDIR/binary/sets .
218 Defaults to
219 .Pa /var/pub/NetBSD .
220 .It EXTRAMK
221 Specifies a whitespace-separated list of files that must be appended to
222 .Pa $DESTDIR/etc/mk.conf .
223 This is useful to add special items to this configuration file.
224 Defaults to nothing.
225 .It INSTALL_PACKAGES
226 A list of packages to automatically install after the
227 .Sy makeroot
228 and before installing
229 .Sy BUILD_PACKAGES .
230 These are also installed within the sandbox created by the
231 .Sy auto
232 target, but before anything is built.
233 Each name must be the full package name, including the tgz suffix.
234 Packages are searched inside
235 .Pa $REAL_PACKAGES/All .
236 Defaults to nothing.
237 .It GENERATE_PKG_SUMMARY
238 If set to
239 .Sq yes ,
240 generate a new
241 .Pa $REAL_PACKAGES/pkg_summary.gz
242 file at the end of every package build by both the
243 .Sq auto
244 and
245 .Sq build
246 targets.
247 .It LOCALBASE
248 Where binary packages get installed.
249 Defaults to
250 .Pa /usr/pkg .
251 .It MKCONF_VARS
252 A list of variable names that will be appended to the generated
253 .Pa /etc/mk.conf
254 file, together with their values set in the configuration file.
255 Its default value contains all variables listed here.
256 .It NETBSD_RELEASE
257 Specifies which version number of
258 .Nx
259 is installed inside the chroot.
260 If set to
261 .Ql no ,
262 no special action is taken (this is useful if the system version inside
263 the chroot matches the outside one).
264 Otherwise, the package
265 .Pa pkgtools/libkver
266 will be installed inside the chroot, in a special purpose
267 prefix whose value can be set in
268 .Pa $DESTDIR/etc/mk.conf
269 via the configuration file
270 with the
271 .Va LIBKVER_STANDALONE_PREFIX
272 variable.
273 The libkver library will be configured inside the chroot, with the symbolic link
274 .Pa $DESTDIR/libkver_osrelease
275 and
276 .Va LD_PRELOAD
277 in default shells environments,
278 so that the
279 .Nx
280 version specified in
281 .Va NETBSD_RELEASE
282 overrides the host system version.
283 See
284 .Xr kver 3
285 for more information.
286 Defaults to
287 .Ql no .
288 .It PKG_DBDIR
289 Location of the packages database.
290 Defaults to
291 .Pa /var/db/pkg .
292 .It PKG_SYSCONFBASE
293 Base directory of configuration files.
294 Defaults to
295 .Pa /usr/pkg/etc .
296 .It PKGSRC_COMPILER
297 List of values specifying the chain of compilers to invoke when building
298 packages.
299 Defaults to
300 .Ql gcc .
301 If you are defining
302 .Va REAL_CCACHE ,
303 remember to prepend
304 .Ql ccache
305 to this variable's value.
306 .It PKGVULNDIR
307 Directory where the
308 .Pa vulnerabilities
309 file will be installed (inside the chroot).
310 Defaults to
311 .Pa /usr/pkg/share .
312 .It REAL_PKGVULNDIR
313 Directory where the system-wide
314 .Pa vulnerabilities
315 file resides (outside the chroot).
316 Defaults to
317 .Pa /usr/pkgsrc/distfiles .
318 .It ROOTSHELL
319 The shell of the root user.
320 Defaults to
321 .Pa /bin/ksh .
322 .It SETS
323 A list of binary sets to be extracted inside
324 .Sy DESTDIR .
325 Defaults to
326 .Ql base.tgz comp.tgz etc.tgz kern-GENERIC.tgz text.tgz .
327 If no kernel is extracted by these sets, an empty
328 .Pa /netbsd
329 file is created inside the chroot.
330 .It SETS_X11
331 A list of binary sets of the X Window system.
332 This has the same behavior
333 as
334 .Sy SETS .
335 If this variable is set to
336 .Ql no ,
337 no X Window is configured inside the chroot
338 jail and no other X variables take effect.
339 Defaults to
340 .Ql xbase.tgz xcomp.tgz xetc.tgz xfont.tgz xserver.tgz .
341 .It SYNC_UMOUNT
342 If set to
343 .Ql yes ,
344 run
345 .Xr sync 8
346 three times after all file systems have been unmounted.
347 Defaults to
348 .Ql no .
349 .It USE_AUDIT_PACKAGES
350 If set to
351 .Ql yes ,
352 let
353 .Nm
354 handle the
355 .Pa vulnerabilities
356 file automatically.
357 This means that it will install the system-wide
358 .Pa vulnerabilities
359 file inside the chroot when needed, keeping both in sync.
360 Defaults to
361 .Ql yes .
362 .It USE_GCC3
363 If set to
364 .Ql yes ,
365 the GNU C Compiler version 3 will be installed inside the chroot
366 environment and used to build all packages, using the
367 .Pa lang/gcc3
368 package.
369 Defaults to
370 .Ql no .
371 .It USE_XPKGWEDGE
372 If set to
373 .Ql yes ,
374 you want xpkgwedge to be compiled and installed automatically inside the
375 chroot.
376 This takes care of setting up
377 .Pa /etc/profile
378 and
379 .Pa /etc/csh.login
380 for xpkgwedge to work.
381 Has no effect if X is unconfigured.
382 Defaults to
383 .Ql yes .
384 .El
385 .Ss Mounted file systems
386 In order to avoid duplicating huge system trees,
387 .Nm
388 takes advantage of file system layers.
389 By default, it uses
390 .Xr mount_null 8 ,
391 which duplicates a file system tree into another directory; although
392 you may want to use
393 .Xr mount_union 8 ,
394 or even
395 .Xr mount_overlay 8 .
396 If the
397 content of these variables is empty, that file system is not mounted.
398 .Pp
399 You can control which layer to use and which options you want with
400 special configuration options, as explained below.
401 .Pp
402 These file systems are mounted before entering the chroot and unmounted
403 after exiting.
404 In order to know if file systems are mounted or not, the
405 program uses a temporary file, called
406 .Pa $DESTDIR/pkg_comp/tmp/mount.stat ,
407 which controls the number of
408 .Nm
409 processes using the chroot environment.
410 If some of them crashes unexpectedly and you notice it does not try
411 to unmount the file systems, this status file may get out of sync.
412 Be sure to check that NO file systems are mounted when issuing a
413 .Sy removeroot .
414 .Bl -tag -width indent
415 .It REAL_CCACHE
416 Specifies where a global ccache directory resides in the real system.
417 Defaults to nothing, which disables the global cache.
418 Keep in mind that this is specially useful to keep the cache across
419 rebuilds of the sandbox, but be very careful if you plan to share a
420 cache directory between different sandboxes, as this can lead to problems.
421 .It REAL_DISTFILES
422 Specifies where distfiles reside in the real system.
423 Defaults to
424 .Pa /usr/pkgsrc/distfiles .
425 .It REAL_DISTFILES_OPTS
426 Mount options.
427 Defaults to
428 .Sy -t null -o rw .
429 .It REAL_PACKAGES
430 Specifies where to build binary packages.
431 This variable is specially useful.
432 Defaults to
433 .Pa /usr/pkgsrc/packages .
434 .It REAL_PACKAGES_OPTS
435 Mount options.
436 Defaults to
437 .Sy -t null -o rw .
438 .It REAL_PKGSRC
439 The pkgsrc tree.
440 This can be useful if you want to use several pkgsrc trees independently.
441 Defaults to
442 .Pa /usr/pkgsrc .
443 .It REAL_PKGSRC_OPTS
444 Mount options.
445 Defaults to
446 .Sy -t null -o ro .
447 .It REAL_SRC
448 The src system tree.
449 Usually useless, but may be needed by some packages, like sysutils/aperture.
450 Defaults to
451 .Pa /usr/src .
452 .It REAL_SRC_OPTS
453 Mount options.
454 Defaults to
455 .Sy -t null -o ro .
456 .It MAKEROOT_HOOKS
457 A whitespace separated list of functions or external scripts to be executed
458 after the sandbox is created.
459 Two arguments are given to each of them:
460 .Ar $DESTDIR ,
461 and the word
462 .Ar makeroot .
463 Defaults to nothing.
464 .It MOUNT_HOOKS
465 A whitespace separated list of functions or external scripts to be executed
466 after file systems are mounted.
467 Two arguments are given to each of them:
468 .Ar $DESTDIR ,
469 and the word
470 .Ar mount .
471 Defaults to nothing.
472 .It UMOUNT_HOOKS
473 A whitespace separated list of functions or external scripts to be executed
474 before file systems are unmounted.
475 Two arguments are given to each of them:
476 .Ar $DESTDIR ,
477 and the word
478 .Ar umount .
479 Defaults to nothing.
480 .El
481 .Sh TARGETS
482 A target specifies what
483 .Nm
484 should do (as in make).
485 The following list describes all supported targets,
486 in the logical order you should call them.
487 .Bl -tag -width indent
488 .It maketemplate
489 Create a sample
490 .Ar conf_file .
491 You should edit it after the creation as you will probably want to change
492 the default configuration, specially paths.
493 .It makeroot
494 Create the chroot environment, based on the specs of the configuration file.
495 This step is required before trying any other, except maketemplate.
496 .It build
497 Builds the specified packages inside the chroot.
498 You can pass the package names as a relative path within pkgsrc or as the
499 basename of the package directory (i.e. omitting the directory name).
500 .It install
501 Install the specified binary packages into the chroot.
502 Package names can contain globs.
503 The package files will be taken from within
504 .Sy REAL_PACKAGES .
505 .It chroot
506 Enters the chroot environment.
507 If no arguments are given,
508 .Va ROOTSHELL
509 is executed, otherwise whatever you typed.
510 If the first argument begins with a word prefixed by
511 .Li pkg_ ,
512 then the
513 .Ql chroot
514 argument can be omitted (it is implied).
515 .It removeroot
516 Remove the entire chroot tree.
517 You should do it with this target because it
518 will take care to umount needed mount points.
519 .It auto
520 This executes several targets automatically, using
521 .Sy AUTO_TARGET
522 as
523 .Sy BUILD_TARGET
524 during the build.
525 The order is: makeroot, build and removeroot.
526 This is useful to create binary packages of several pkgsrc and their
527 dependencies automatically.
528 For this to be useful, you need to set
529 .Sy REAL_PACKAGES
530 and use
531 .Sy AUTO_PACKAGES
532 or pass package names through the command line.
533 .Pp
534 If the magic word
535 .Ql resume
536 is passed as the unique argument to this target,
537 .Nm
538 will attempt to resume a previous automatic build for the given configuration.
539 .El
540 .Sh NOTES
541 This program uses nullfs to create virtual copies of real trees inside the
542 chroot environment.
543 .Pp
544 You need to install the
545 .Pa security/audit-packages
546 package in the host system (and have an up to date vulnerabilities database)
547 if you want security checks to work inside the
548 chroot environment.
549 .Sh SEE ALSO
550 .Xr pkg_delete 1 ,
551 .Xr pkgsrc 7 ,
552 .Xr mount_null 8 ,
553 .Xr sync 8 ,
554 .Xr sysctl 8
555 .Sh AUTHORS
556 .An Julio M. Merino Vidal Aq jmmv@NetBSD.org