| blob | efd78ae837339797932c8f78ae9c247c40aae5e6 |
1 # ctest_harness\r
2 **ctest_harness** is a small and portable unit test framework for C.\r
3 \r
4 ## Features\r
5 - **Easy to integrate**\r
6 \r
7 Having only one header file and source file, integrating it into your build system is trivial.\r
8 - **Small, Portable**\r
9 - C99 compliant\r
10 - Self-contained, no external dependencies\r
11 - Easy to compile and uses POSIX make\r
12 - Should compile without warnings using `-Wall -Wextra -Wpedantic`\r
13 - No dynamic allocation.\r
14 - **Timing information**\r
15 \r
16 Provides wall clock timing information to help diagnose performance regressions.\r
17 - **Test suites**\r
18 \r
19 Organize your tests into test suites.\r
20 - **Handy assertion macros**\r
21 \r
22 Assertions are provided for testing equality and inequality of integral data types, strings and double floating point.\r
23 - **Pseudo-random number generator**\r
24 \r
25 32-bit PRNG based on [xoroshiro64**](https://prng.di.unimi.it/) used for seed generation and to aid the creation of randomized tests.\r
26 \r
27 ## Getting Started\r
28 **ctest_harness** is designed to be easy to use. Just add harness.c to your sources, include harness.h, and you're good to go. Here's a basic example\r
29 ```c\r
30 #include "harness.h"\r
31 \r
32 static int x_should_equal_1(void *user_data) {\r
33 int x = 1;\r
34 \r
35 ASSERT_EQ(1, x);\r
36 \r
37 return HARNESS_PASS;\r
38 }\r
39 \r
40 int main(void) {\r
41 struct harness_cfg cfg = {0};\r
42 int ret = EXIT_SUCCESS;\r
43 \r
44 if (harness_init(&cfg) == HARNESS_FAIL) {\r
45 return EXIT_FAILURE;\r
46 }\r
47 \r
48 (void)harness_add_test("my_suite", "x_should_equal_1",x_should_equal_1, NULL, NULL, NULL);\r
49 ret = harness_run(NULL);\r
50 harness_destroy(&cfg);\r
51 \r
52 return ret;\r
53 }\r
54 ```\r
55 \r
56 ## Assertions\r
57 Assertions are a fundamental part of testing. **ctest_harness** provides two levels of assertions: (1) fatal and (2) non-fatal. Fatal assertions stop\r
58 the execution of the test resulting in a FAIL verdict, while non-fatal assertions as the name implies don't stop the execution.\r
59 \r
60 Let's say you want to test two values for equality:\r
61 ```c\r
62 int your_test_function(void *user_data) {\r
63 (void)user_data;\r
64 \r
65 int a = 5;\r
66 int b = 10;\r
67 \r
68 ASSERT_EQ(a, b);\r
69 \r
70 return HARNESS_PASS;\r
71 }\r
72 ```\r
73 ```\r
74 ERROR> srcfile.c:6: assertion failed: a == b\r
75 ```\r
76 The non-fatal variants are prefixed with `EXPECT_` instead of `ASSERT_`.\r
77 \r
78 ## Pseudo-Random numbers\r
79 Being able to randomize tests is a great way to increase coverage of your tests. The drawback of randomized tests is reproducibility: a test failed,\r
80 but if it's randomized then how do I reproduce it? That's where seeding comes into play, by initializing the PRNG with the same seed you can\r
81 reproduce the exact same scenario. Every time the test harness runs an unsigned 32-bit seed is generated and printed in hexadecimal notation so that\r
82 you can later use it to re-run the tests with the same output. If you want to set a specific seed you can do that by setting the `seed` field in the\r
83 `struct harness_cfg`.\r
84 \r
85 The API is the following:\r
86 ```c\r
87 void random_init(void);\r
88 ```\r
89 Initialize the state with the value of time(0) is used.\r
90 \r
91 ```c\r
92 uint32_t random_next(void);\r
93 ```\r
94 Generate a 32-bit random value\r
95 \r
96 ```c\r
97 uint32_t random_bounded(uint32_t range);\r
98 ```\r
99 Generate a 32-bit random value in the interval [0, range)\r
100 \r
101 ```c\r
102 float random_float(void);\r
103 ```\r
104 Generate a random float in the interval [0, 1)\r
105 \r
106 \r
107 ## Setting up the test harness\r
108 Before adding and executing test cases you need to initilize the test harness. This is done by first defining a variable of type struct `harness_cfg`\r
109 and calling `harness_init`:\r
110 ```c\r
111 struct harness_cfg cfg = {\r
112 .setup = setup,\r
113 .teardown = teardown,\r
114 };\r
115 \r
116 harness_init(&cfg);\r
117 ```\r
118 This is also the place where we have to define a global fixture setup and teardown, they must follow the following prototype (of course the name can\r
119 be changed):\r
120 ```c\r
121 void setup_function(void *user_data);\r
122 void teardown_function(void *user_data);\r
123 ```\r
124 \r
125 If they are not needed they can be set to NULL.\r
126 \r
127 Once we are done with our tests we have to destroy the test harness:\r
128 ```c\r
129 harness_destroy(&cfg);\r
130 ```\r
131 \r
132 ## Tests and Suites\r
133 Let's look at how to structure testcases for **ctest_harness**. Each testcase should be a separate function with the following prototype:\r
134 ```c\r
135 int my_test(void *user_data);\r
136 ```\r
137 Of course the name of the test does not matter it's only for your internal use and convenience.\r
138 \r
139 The possible return values of a test case are four:\r
140 \r
141 | | |\r
142 |---:|---|\r
143 |`HARNESS_PASS`|The test passed|\r
144 |`HARNESS_FAIL`|The test failed, if an assertion fails this is the result|\r
145 |`HARNESS_SKIP`|The test was skipped for some reason, this can be used in situations where the test is not applicable.|\r
146 |`HARNESS_ERROR`|There was an error during the test case execution. This type of error is used when the error does not have to do with the scope of\r
147 the test. For example your test writes and reads data to and from SRAM via SPI but the SPI driver cannot be initialized|\r
148 \r
149 It is recommended that each test scenario should be implemented in a different function.\r
150 \r
151 ### Adding a test case\r
152 Once you have defined a test case, or more than one, it's time to add them to a test suite. There are two ways you can do that: you can either add\r
153 them one by one using the `harness_add_test` function:\r
154 \r
155 ```c\r
156 harness_add_test(\r
157 "my_suite", /* suite name */\r
158 "my_test", /* test name */\r
159 my_test, /* test function */\r
160 NULL, /* fixture setup */\r
161 NULL, /* fixture teardown */\r
162 NULL /* user data */\r
163 );\r
164 ```\r
165 or you can add them by declaring an array of struct `harness_test` and then feeding that array to `harness_add_suite`:\r
166 ```c\r
167 struct harness_test tests[] = {\r
168 {"my_test", my_test, NULL, NULL, NULL},\r
169 {NULL, NULL, NULL, NULL, NULL}\r
170 };\r
171 \r
172 harness_add_suite("my_suite", tests);\r
173 ```\r
174 Note that the NULL entry at the end is used to tell the test runner when the array is over.\r
175 \r
176 ### Fixture setup and teardown\r
177 **ctest_harness** has two levels of fixtures: *global* and *test specific* fixtures. If global setup and teardown functions are provided to the\r
178 `harness_cfg` structure, the setup function will be executed before all tests are run. Similarly if a global teardown function is provided it will be\r
179 executed at the end of all the test cases. Test specific fixtures will be executed before (setup) and after (teardown) each test case.\r
180 \r
181 These functions are commonly used to initialize and destroy resources used by all the test cases or by a specific test. For example:\r
182 ```c\r
183 static void setup(void *user_data) {\r
184 char *buf = (char *)user_data;\r
185 buf = malloc(1024);\r
186 }\r
187 \r
188 static void teardown(void *user_data) {\r
189 char *buf = (char *)user_data;\r
190 free(buf);\r
191 }\r
192 \r
193 static void test_setup(void *user_data) {\r
194 char *buf = (char *)user_data;\r
195 FILE *fp = fopen("file.txt", "r");\r
196 fgets(buf, 1024, fp);\r
197 fclose(fp);\r
198 }\r
199 \r
200 static void test_teardown(void *user_data) {\r
201 (void)user_data;\r
202 }\r
203 \r
204 static int my_test(void *user_data) {\r
205 char* buf = (char *)user_data;\r
206 ASSERT_STREQ(buf, "Hello, world!");\r
207 \r
208 return HARNESS_PASS;\r
209 }\r
210 \r
211 int main(void) {\r
212 struct harness_cfg cfg = {\r
213 .setup = setup,\r
214 .teardown = teardown,\r
215 };\r
216 char *buf = 0;\r
217 \r
218 harness_init(&cfg);\r
219 harness_add_test("my_suite", "my_test", my_test, test_setup, test_teardown, buf);\r
220 }\r
221 ```\r
222 \r
223 ### Running the tests\r
224 Once you've added your tests and suites, it's time to call `harness_run`, its prototype is\r
225 ```c\r
226 int harness_run(void *user_data);\r
227 ```\r
228 The return value will be `EXIT_SUCCESS` if all tests pass,or `EXIT_FAILURE` if any test has failed. This makes it particularly suitable for returning\r
229 directly from your main() function.\r
230 \r
231 ### User data\r
232 Up until now we've skipped talking about the `user_data` parameters, but it's time to address it.\r
233 \r
234 If a global setup and/or teardown function has been provided, the `user_data` parameter passed to `harness_run` is passed to the setup and/or teardown\r
235 fixtures.\r
236 \r
237 If a test specific setup and/or teardown function has been provided, the `user_data` field of the `struct harness_test` is passed to the setup and/or\r
238 teardown fixtures as well as the test function.\r
239 \r
240 ## Static analysis\r
241 Static analysis on the code base is done by using clang's static analyzer run through `scan-build.sh` which wraps the\r
242 `scan-build` utility. The checkers used are part of the\r
243 [Experimental Checkers](https://releases.llvm.org/12.0.0/tools/clang/docs/analyzer/checkers.html#alpha-checkers)\r
244 (aka *alpha* checkers):\r
245 \r
246 * `alpha.security`\r
247 * `alpha.core.CastSize`\r
248 * `alpha.core.CastToStruct`\r
249 * `alpha.core.IdenticalExpr`\r
250 * `alpha.core.PointerArithm`\r
251 * `alpha.core.PointerSub`\r
252 * `alpha.core.SizeofPtr`\r
253 * `alpha.core.TestAfterDivZero`\r
254 * `alpha.unix`\r
255 \r
256 ## License\r
257 BSD 2-Clause FreeBSD License, see LICENSE.\r