| blob | 942302dc89b90178bcabc050dc792be2cf687d92 |
1 .\" $NetBSD: atf-report.1,v 1.3 2014/12/10 04:38:03 christos Exp $
2 .\"
3 .\"
4 .\" Automated Testing Framework (atf)
5 .\"
6 .\" Copyright (c) 2007 The NetBSD Foundation, Inc.
7 .\" All rights reserved.
8 .\"
9 .\" Redistribution and use in source and binary forms, with or without
10 .\" modification, are permitted provided that the following conditions
11 .\" are met:
12 .\" 1. Redistributions of source code must retain the above copyright
13 .\" notice, this list of conditions and the following disclaimer.
14 .\" 2. Redistributions in binary form must reproduce the above copyright
15 .\" notice, this list of conditions and the following disclaimer in the
16 .\" documentation and/or other materials provided with the distribution.
17 .\"
18 .\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
19 .\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
20 .\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
21 .\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
22 .\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
23 .\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
24 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
25 .\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
26 .\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
27 .\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
28 .\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
29 .\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
30 .\"
31 .Dd December 16, 2011
32 .Dt ATF-REPORT 1
33 .Os
34 .Sh NAME
35 .Nm atf-report
36 .Nd transforms the output of atf-run to different formats
37 .Sh SYNOPSIS
38 .Nm
39 .Op Fl o Ar fmt1:path1 Op .. Fl o Ar fmtN:pathN
40 .Nm
41 .Fl h
42 .Sh DESCRIPTION
43 .Nm
44 reads the output of
45 .Nm atf-run
46 and transforms it to different formats.
47 Some of these are user-friendly and others are machine-parseable, which
48 opens a wide range of possibilities to analyze the results of a test
49 suite's execution.
50 See
51 .Sx Output formats
52 below for more details on which these formats are.
53 .Pp
54 In the first synopsis form,
55 .Nm
56 reads the output of
57 .Nm atf-run
58 through its standard input and, if no
59 .Fl o
60 options are given, prints a user-friendly report on its standard
61 output using the
62 .Sq ticker
63 format.
64 If
65 .Fl o
66 options are provided (more than one are allowed), they specify the complete
67 list of reports to generate.
68 They are all generated simultaneously, and for obvious reasons, two reports
69 cannot be written to the same file.
70 Note that the default output is suppressed when
71 .Fl o
72 is provided.
73 .Pp
74 In the second synopsis form,
75 .Nm
76 will print information about all supported options and their purpose.
77 .Pp
78 The following options are available:
79 .Bl -tag -width XoXfmtXpathXX
80 .It Fl h
81 Shows a short summary of all available options and their purpose.
82 .It Fl o Ar fmt:path
83 Adds a new output format.
84 .Ar fmt
85 is one of the formats described later on in
86 .Sx Output formats .
87 .Ar path
88 specifies where the report will be written to.
89 Depending on the chosen format, this may refer to a single file or to
90 a directory.
91 For those formats that write to a single file, specifying a
92 .Sq -
93 as the path will redirect the report to the standard output.
94 .El
95 .Ss Output formats
96 The following output formats are allowed:
97 .Bl -tag -width tickerXX
98 .It csv
99 A machine-parseable Comma-Separated Values (CSV) file.
100 This file contains the results for all test cases and test programs.
101 Test cases are logged using the following syntax:
102 .Bd -literal -offset indent
103 tc, duration, test-program, test-case, result[, reason]
104 .Ed
105 .Pp
106 The
107 .Sq result
108 field for test cases is always one of
109 .Sq passed ,
110 .Sq skipped
111 or
112 .Sq failed .
113 The last two are always followed by a reason.
114 .Pp
115 Test programs are logged with the following syntax:
116 .Bd -literal -offset indent
117 tp, duration, test-program, result[, reason]
118 .Ed
119 .Pp
120 In this case, the
121 .Sq result
122 can be one of:
123 .Sq passed ,
124 which denotes test programs that ran without any failure;
125 .Sq failed ,
126 which refers to test programs in which one or more test cases failed;
127 or
128 .Sq bogus ,
129 which mentions those test programs that failed to execute by some reason.
130 The reason field is only available in the last case.
131 .Pp
132 The time required to execute each test case and test program is
133 also provided.
134 You should not rely on the order of the entries in the resulting output.
135 .It ticker
136 A user-friendly report that shows the progress of the test suite's
137 execution as it operates.
138 This type of report should always be redirected to a virtual terminal,
139 not a file, as it may use control sequences that will make the output
140 unreadable in regular files.
141 .It xml
142 A report contained in a single XML file.
143 Ideal for later processing with
144 .Xr xsltproc 1
145 to generate nice HTML reports.
146 .El
147 .Sh EXAMPLES
148 The most simple way of running a test suite is to pipe the output of
149 .Nm atf-run
150 through
151 .Nm
152 without any additional flags.
153 This will use the default output format, which is suitable to most users:
154 .Bd -literal -offset indent
155 atf-run | atf-report
156 .Ed
157 .Pp
158 In some situations, it may be interesting to get a machine-parseable file
159 aside from the standard report.
160 This can be done as follows:
161 .Bd -literal -offset indent
162 atf-run | atf-report -o csv:testsuite.csv -o ticker:-
163 .Ed
164 .Pp
165 Or if the standard report is not desired, thus achieving completely silent
166 operation:
167 atf-run | atf-report -o csv:testsuite.csv
168 .Sh SEE ALSO
169 .Xr atf-run 1 ,
170 .Xr atf 7