| blob | bf7837f59e4ec088bb4c563f3828696e1fe0258d |
1 # ctest_harness\r
2 **ctest_harness** is a small and portable unit test framework for C.\r
3 \r
4 ## Features\r
5 - **Easy to integrate**\r
6 \r
7 Having only one header file and source file, integrating it into your build system is trivial.\r
8 - **Small, Portable**\r
9 - C99 compliant\r
10 - Self-contained, no external dependencies\r
11 - Easy to compile and uses POSIX make\r
12 - Should compile without warnings using `-Wall -Wextra -Wpedantic`\r
13 - No dynamic allocation.\r
14 - **Timing information**\r
15 \r
16 Provides wall clock timing information to help diagnose performance regressions.\r
17 - **Test suites**\r
18 \r
19 Organize your tests into test suites.\r
20 - **Handy assertion macros**\r
21 \r
22 Assertions are provided for testing equality and inequality of integral data types, strings and double floating point.\r
23 - **Pseudo-random number generator**\r
24 \r
25 32-bit PRNG based on [xoroshiro64**](https://prng.di.unimi.it/) used for seed generation and to aid the creation of randomized tests.\r
26 \r
27 ## Getting Started\r
28 **ctest_harness** is designed to be easy to use. Just add harness.c to your sources, include harness.h, and you're good to go. Here's a basic example\r
29 ```c\r
30 #include "harness.h"\r
31 \r
32 static int x_should_equal_1(void *user_data) {\r
33 int x = 1;\r
34 \r
35 ASSERT_EQ(1, x);\r
36 \r
37 return HARNESS_PASS;\r
38 }\r
39 \r
40 int main(void) {\r
41 struct harness_cfg cfg = {0};\r
42 int ret = EXIT_SUCCESS;\r
43 \r
44 if (harness_init(&cfg) == HARNESS_FAIL) {\r
45 return EXIT_FAILURE;\r
46 }\r
47 \r
48 (void)harness_add_test("my_suite", "x_should_equal_1",x_should_equal_1, NULL, NULL, NULL);\r
49 ret = harness_run(NULL);\r
50 harness_destroy(&cfg);\r
51 \r
52 return ret;\r
53 }\r
54 ```\r
55 \r
56 ## Assertions\r
57 Assertions are a fundamental part of testing. **ctest_harness** provides two levels of assertions: (1) fatal and (2) non-fatal. Fatal assertions stop\r
58 the execution of the test resulting in a FAIL verdict, while non-fatal assertions as the name implies don't stop the execution.\r
59 \r
60 Let's say you want to test two values for equality:\r
61 ```c\r
62 int your_test_function(void *user_data) {\r
63 (void)user_data;\r
64 \r
65 int a = 5;\r
66 int b = 10;\r
67 \r
68 ASSERT_EQ(a, b);\r
69 \r
70 return HARNESS_PASS;\r
71 }\r
72 ```\r
73 ```\r
74 ERROR> srcfile.c:6: assertion failed: a == b\r
75 ```\r
76 The non-fatal variants are prefixed with `EXPECT_` instead of `ASSERT_`.\r
77 \r
78 ## Pseudo-Random numbers\r
79 Being able to randomize tests is a great way to increase coverage of your tests. The drawback of randomized tests is reproducibility: a test failed,\r
80 but if it's randomized then how do I reproduce it? That's where seeding comes into play, by initializing the PRNG with the same seed you can\r
81 reproduce the exact same scenario. Every time the test harness runs an unsigned 32-bit seed is generated and printed in hexadecimal notation so that\r
82 you can later use it to re-run the tests with the same output. If you want to set a specific seed you can do that by setting the `seed` field in the\r
83 `struct harness_cfg`.\r
84 \r
85 The API is the following:\r
86 ```c\r
87 void random_init(void);\r
88 ```\r
89 Initialize the state with the value of time(0) is used.\r
90 \r
91 ```c\r
92 uint32_t random_next(void);\r
93 ```\r
94 Generate a 32-bit random value\r
95 \r
96 ```c\r
97 uint32_t random_bounded(uint32_t range);\r
98 ```\r
99 Generate a 32-bit random value in the interval [0, range)\r
100 \r
101 ```c\r
102 float random_float(void);\r
103 ```\r
104 Generate a random float in the interval [0, 1)\r
105 \r
106 \r
107 ## Setting up the test harness\r
108 Before adding and executing test cases you need to initilize the test harness. This is done by first defining a variable of type struct `harness_cfg`\r
109 and calling `harness_init`:\r
110 ```c\r
111 struct harness_cfg cfg = {\r
112 .setup = setup,\r
113 .teardown = teardown,\r
114 .iterations = 1, /* if not set or set to 0, it defaults to 1 */\r
115 };\r
116 \r
117 harness_init(&cfg);\r
118 ```\r
119 This is also the place where we have to define a global fixture setup and teardown, they must follow the following prototype (of course the name can\r
120 be changed):\r
121 ```c\r
122 int setup_function(void *user_data);\r
123 void teardown_function(void *user_data);\r
124 ```\r
125 If they are not needed they can be set to `NULL`.\r
126 \r
127 Once we are done with our tests we have to destroy the test harness:\r
128 ```c\r
129 harness_destroy(&cfg);\r
130 ```\r
131 \r
132 ## Tests and Suites\r
133 Let's look at how to structure testcases for **ctest_harness**. Each testcase should be a separate function with the following prototype:\r
134 ```c\r
135 int my_test(void *user_data);\r
136 ```\r
137 Of course the name of the test does not matter it's only for your internal use and convenience.\r
138 \r
139 The possible return values of a test case are four:\r
140 \r
141 | | |\r
142 |---:|---|\r
143 |`HARNESS_PASS`|The test passed|\r
144 |`HARNESS_FAIL`|The test failed, if an assertion fails this is the result|\r
145 |`HARNESS_SKIP`|The test was skipped for some reason, this can be used in situations where the test is not applicable.|\r
146 |`HARNESS_ERROR`|There was an error during the test case execution. This type of error is used when the error does not have to do with the scope of\r
147 the test. For example your test writes and reads data to and from SRAM via SPI but the SPI driver cannot be initialized|\r
148 \r
149 It is recommended that each test scenario should be implemented in a different function.\r
150 \r
151 ### Adding a test case\r
152 Once you have defined a test case, or more than one, it's time to add them to a test suite. There are two ways you can do that: you can either add\r
153 them one by one using the `harness_add_test` function:\r
154 \r
155 ```c\r
156 harness_add_test(\r
157 "my_suite", /* suite name */\r
158 "my_test", /* test name */\r
159 my_test, /* test function */\r
160 NULL, /* fixture setup */\r
161 NULL, /* fixture teardown */\r
162 NULL /* user data */\r
163 );\r
164 ```\r
165 or you can add them by declaring an array of struct `harness_test` and then feeding that array to `harness_add_suite`:\r
166 ```c\r
167 struct harness_test tests[] = {\r
168 {"my_test", my_test, NULL, NULL, NULL},\r
169 {NULL, NULL, NULL, NULL, NULL}\r
170 };\r
171 \r
172 harness_add_suite("my_suite", tests);\r
173 ```\r
174 Note that the NULL entry at the end is used to tell the test runner when the array is over.\r
175 \r
176 ### Fixture setup and teardown\r
177 **ctest_harness** has two levels of fixtures: *global* and *test specific* fixtures. If global setup and teardown functions are provided to the\r
178 `harness_cfg` structure, the setup function will be executed before all tests are run. Similarly if a global teardown function is provided it will be\r
179 executed at the end of all the test cases. Test specific fixtures will be executed before (setup) and after (teardown) each test case.\r
180 \r
181 These functions are commonly used to initialize and destroy resources used by all the test cases or by a specific test. For example:\r
182 ```c\r
183 static int setup(void *user_data) {\r
184 char *buf = (char *)user_data;\r
185 \r
186 buf = malloc(1024);\r
187 if (buf == NULL) {\r
188 return (HARNESS_ERROR);\r
189 }\r
190 \r
191 return (HARNESS_PASS);\r
192 }\r
193 \r
194 static void teardown(void *user_data) {\r
195 char *buf = (char *)user_data;\r
196 free(buf);\r
197 }\r
198 \r
199 static int test_setup(void *user_data) {\r
200 char *buf = (char *)user_data;\r
201 FILE *fp = fopen("file.txt", "r");\r
202 if (fp == NULL) {\r
203 return (HARNESS_ERROR);\r
204 }\r
205 \r
206 fgets(buf, 1024, fp);\r
207 fclose(fp);\r
208 \r
209 return (HARNESS_SUCCESS);\r
210 }\r
211 \r
212 static void test_teardown(void *user_data) {\r
213 (void)user_data;\r
214 }\r
215 \r
216 static int my_test(void *user_data) {\r
217 char* buf = (char *)user_data;\r
218 ASSERT_STREQ(buf, "Hello, world!");\r
219 \r
220 return HARNESS_PASS;\r
221 }\r
222 \r
223 int main(void) {\r
224 struct harness_cfg cfg = {\r
225 .setup = setup,\r
226 .teardown = teardown,\r
227 };\r
228 char *buf = 0;\r
229 \r
230 harness_init(&cfg);\r
231 harness_add_test("my_suite", "my_test", my_test, test_setup, test_teardown, buf);\r
232 }\r
233 ```\r
234 \r
235 The return value of the test specific setup function will be checked before executing the test case itself, if a value different from `HARNESS_SUCCESS`\r
236 is returned then the execution will stop without calling the actual test function.\r
237 \r
238 \r
239 ### Running the tests\r
240 Once you've added your tests and suites, it's time to call `harness_run`, its prototype is\r
241 ```c\r
242 int harness_run(void *user_data);\r
243 ```\r
244 The return value will be `EXIT_SUCCESS` if all tests pass,or `EXIT_FAILURE` if any test has failed. This makes it particularly suitable for returning\r
245 directly from your main() function.\r
246 \r
247 ### User data\r
248 Up until now we've skipped talking about the `user_data` parameters, but it's time to address it.\r
249 \r
250 If a global setup and/or teardown function has been provided, the `user_data` parameter passed to `harness_run` is passed to the setup and/or teardown\r
251 fixtures.\r
252 \r
253 If a test specific setup and/or teardown function has been provided, the `user_data` field of the `struct harness_test` is passed to the setup and/or\r
254 teardown fixtures as well as the test function.\r
255 \r
256 ## Static analysis\r
257 Static analysis on the code base is done by using clang's static analyzer run through `scan-build.sh` which wraps the\r
258 `scan-build` utility. The checkers used are part of the\r
259 [Experimental Checkers](https://releases.llvm.org/12.0.0/tools/clang/docs/analyzer/checkers.html#alpha-checkers)\r
260 (aka *alpha* checkers):\r
261 \r
262 * `alpha.security`\r
263 * `alpha.core.CastSize`\r
264 * `alpha.core.CastToStruct`\r
265 * `alpha.core.IdenticalExpr`\r
266 * `alpha.core.PointerArithm`\r
267 * `alpha.core.PointerSub`\r
268 * `alpha.core.SizeofPtr`\r
269 * `alpha.core.TestAfterDivZero`\r
270 * `alpha.unix`\r
271 \r
272 ## License\r
273 BSD 2-Clause FreeBSD License, see LICENSE.\r