3 Network Working Group L. Degioanni
4 Internet-Draft F. Risso
5 Expires: August 30, 2004 Politecnico di Torino
9 PCAP New Generation Dump File Format
14 This document is an Internet-Draft and is in full conformance with
15 all provisions of Section 10 of RFC2026.
17 Internet-Drafts are working documents of the Internet Engineering
18 Task Force (IETF), its areas, and its working groups. Note that other
19 groups may also distribute working documents as Internet-Drafts.
21 Internet-Drafts are draft documents valid for a maximum of six months
22 and may be updated, replaced, or obsoleted by other documents at any
23 time. It is inappropriate to use Internet-Drafts as reference
24 material or to cite them other than as "work in progress."
26 The list of current Internet-Drafts can be accessed at http://
27 www.ietf.org/ietf/1id-abstracts.txt.
29 The list of Internet-Draft Shadow Directories can be accessed at
30 http://www.ietf.org/shadow.html.
32 This Internet-Draft will expire on August 30, 2004.
36 Copyright (C) The Internet Society (2004). All Rights Reserved.
40 This document describes a format to dump captured packets on a file.
41 This format is extensible and it is currently proposed for
42 implementation in the libpcap/WinPcap packet capture library.
55 Degioanni & Risso Expires August 30, 2004 [Page 1]
57 Internet-Draft PCAP New Generation Dump File Format March 2004
62 1. Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . 3
63 2. General File Structure . . . . . . . . . . . . . . . . . . . . 4
64 2.1 General Block Structure . . . . . . . . . . . . . . . . . . . 4
65 2.2 Block Types . . . . . . . . . . . . . . . . . . . . . . . . . 5
66 2.3 Block Hierarchy and Precedence . . . . . . . . . . . . . . . . 5
67 2.4 Data format . . . . . . . . . . . . . . . . . . . . . . . . . 6
68 3. Block Definition . . . . . . . . . . . . . . . . . . . . . . . 8
69 3.1 Section Header Block (mandatory) . . . . . . . . . . . . . . . 8
70 3.2 Interface Description Block (mandatory) . . . . . . . . . . . 9
71 3.3 Packet Block (optional) . . . . . . . . . . . . . . . . . . . 13
72 3.4 Simple Packet Block (optional) . . . . . . . . . . . . . . . . 15
73 3.5 Name Resolution Block (optional) . . . . . . . . . . . . . . . 16
74 3.6 Interface Statistics Block (optional) . . . . . . . . . . . . 18
75 4. Options . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
76 5. Experimental Blocks (deserved to a further investigation) . . 23
77 5.1 Other Packet Blocks (experimental) . . . . . . . . . . . . . . 23
78 5.2 Compression Block (experimental) . . . . . . . . . . . . . . . 23
79 5.3 Encryption Block (experimental) . . . . . . . . . . . . . . . 23
80 5.4 Fixed Length Block (experimental) . . . . . . . . . . . . . . 24
81 5.5 Directory Block (experimental) . . . . . . . . . . . . . . . . 25
82 5.6 Traffic Statistics and Monitoring Blocks (experimental) . . . 25
83 5.7 Event/Security Block (experimental) . . . . . . . . . . . . . 25
84 6. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 27
85 7. Most important open issues . . . . . . . . . . . . . . . . . . 28
86 Intellectual Property and Copyright Statements . . . . . . . . 29
111 Degioanni & Risso Expires August 30, 2004 [Page 2]
113 Internet-Draft PCAP New Generation Dump File Format March 2004
118 The problem of exchanging packet traces becomes more and more
119 critical every day; unfortunately, no standard solutions exist for
120 this task right now. One of the most accepted packet interchange
121 formats is the one defined by libpcap, which is rather old and does
122 not fit for some of the nowadays applications especially in terms of
125 This document proposes a new format for dumping packet traces. The
126 following goals are being pursued:
128 o Extensibility: aside of some common functionalities, third parties
129 should be able to enrich the information embedded in the file with
130 proprietary extensions, which will be ignored by tools that are
131 not able to understand them.
133 o Portability: a capture trace must contain all the information
134 needed to read data independently from network, hardware and
135 operating system of the machine that made the capture.
137 o Merge/Append data: it should be possible to add data at the end of
138 a given file, and the resulting file must still be readable.
167 Degioanni & Risso Expires August 30, 2004 [Page 3]
169 Internet-Draft PCAP New Generation Dump File Format March 2004
172 2. General File Structure
174 2.1 General Block Structure
176 A capture file is organized in blocks, that are appended one to
177 another to form the file. All the blocks share a common format, which
178 is shown in Figure 1.
181 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
182 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
184 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
185 | Block Total Length |
186 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
188 / /* variable length, aligned to 32 bits */ /
189 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
190 | Block Total Length |
191 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
193 Figure 1: Basic block structure.
195 The fields have the following meaning:
197 o Block Type (32 bits): unique value that identifies the block.
198 Values whose Most Significant Bit (MSB) is equal to 1 are reserved
199 for local use. They allow to save private data to the file and to
200 extend the file format.
202 o Block Total Length: total size of this block, in bytes. For
203 instance, a block that does not have a body has a length of 12
206 o Block Body: content of the block.
208 o Block Total Length: total size of this block, in bytes. This field
209 is duplicated for permitting backward file navigation.
211 This structure, shared among all blocks, makes easy to process a file
212 and to skip unneeded or unknown blocks. Blocks can be nested one
213 inside the others (NOTE: needed?). Some of the blocks are mandatory,
214 i.e. a dump file is not valid if they are not present, other are
217 The structure of the blocks allows to define other blocks if needed.
218 A parser that does non understand them can simply ignore their
223 Degioanni & Risso Expires August 30, 2004 [Page 4]
225 Internet-Draft PCAP New Generation Dump File Format March 2004
230 The currently defined blocks are the following:
232 1. Section Header Block: it defines the most important
233 characteristics of the capture file.
235 2. Interface Description Block: it defines the most important
236 characteristics of the interface(s) used for capturing traffic.
238 3. Packet Block: it contains a single captured packet, or a portion
241 4. Simple Packet Block: it contains a single captured packet, or a
242 portion of it, with only a minimal set of information about it.
244 5. Name Resolution Block: it defines the mapping from numeric
245 addresses present in the packet dump and the canonical name
248 6. Capture Statistics Block: it defines how to store some
249 statistical data (e.g. packet dropped, etc) which can be useful
250 to undestand the conditions in which the capture has been made.
252 7. Compression Marker Block: TODO
254 8. Encryption Marker Block: TODO
256 9. Fixed Length Marker Block: TODO
258 The following blocks instead are considered interesting but the
259 authors believe that they deserve more in-depth discussion before
262 1. Further Packet Blocks
266 3. Traffic Statistics and Monitoring Blocks
268 4. Alert and Security Blocks
270 TODO Currently standardized Block Type codes are specified in
273 2.3 Block Hierarchy and Precedence
275 The file must begin with a Section Header Block. However, more than
279 Degioanni & Risso Expires August 30, 2004 [Page 5]
281 Internet-Draft PCAP New Generation Dump File Format March 2004
284 one Section Header Block can be present on the dump, each one
285 covering the data following it till the next one (or the end of
286 file). A Section includes the data delimited by two Section Header
287 Blocks (or by a Section Header Block and the end of the file),
288 including the first Section Header Block.
290 In case an application cannot read a Section because of different
291 version number, it must skip everything until the next Section Header
292 Block. Note that, in order to properly skip the blocks until the next
293 section, all blocks must have the fields Type and Length at the
294 beginning. This is a mandatory requirement that must be maintained in
295 future versions of the block format.
297 Figure 2 shows two valid files: the first has a typical
298 configuration, with a single Section Header that covers the whole
299 file. The second one contains three headers, and is normally the
300 result of file concatenation. An application that understands only
301 version 1.0 of the file format skips the intermediate section and
302 restart processing the packets after the third Section Header.
304 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
306 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
307 Typical configuration with a single Section Header Block
310 |-- 1st Section --|-- 2nd Section --|-- 3rd Section --|
312 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
313 | SHB v1.0 | Data | SHB V1.1 | Data | SHB V1.0 | Data |
314 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
315 Configuration with three different Section Header Blocks
317 Figure 2: File structure example: the Section Header Block.
319 NOTE: TO BE COMPLETED with some examples of other blocks
323 Data contained in each section will always be saved according to the
324 characteristics (little endian / big endian) of the dumping machine.
325 This refers to all fields that are saved as numbers and that span
326 over two or more bytes.
328 The approach of having each section saved in the native format of the
329 generating host is more efficient because it avoids translation of
330 data when reading / writing on the host itself, which is the most
331 common case when generating/processing capture dumps.
335 Degioanni & Risso Expires August 30, 2004 [Page 6]
337 Internet-Draft PCAP New Generation Dump File Format March 2004
340 TODO Probably we have to specify something more here. Is what we're
341 saying enough to avoid any kind of ambiguity?.
391 Degioanni & Risso Expires August 30, 2004 [Page 7]
393 Internet-Draft PCAP New Generation Dump File Format March 2004
398 This section details the format of the body of the blocks currently
401 3.1 Section Header Block (mandatory)
403 The Section Header Block is mandatory. It identifies the beginning of
404 a section of the capture dump file. Its format is shown in Figure 3.
407 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
408 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
410 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
412 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
414 / Options (variable) /
416 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
418 Figure 3: Section Header Block format.
420 The meaning of the fields is:
422 o Magic: magic number, whose value is the hexadecimal number
423 0x1A2B3C4D. This number can be used to distinguish section that
424 have been saved on little-endian machines from the one saved on
427 o Major: number of the current mayor version of the format. Current
430 o Minor: number of the current minor version of the format. Current
433 o Options: optionally, a list of options (formatted according to the
434 rules defined in Section 4) can be present.
436 Aside form the options defined in Section 4, the following options
437 are valid within this block:
439 +----------------+----------------+----------------+----------------+
440 | Name | Code | Length | Description |
441 +----------------+----------------+----------------+----------------+
442 | Hardware | 2 | variable | An ascii |
447 Degioanni & Risso Expires August 30, 2004 [Page 8]
449 Internet-Draft PCAP New Generation Dump File Format March 2004
452 | | | | containing the |
453 | | | | description of |
454 | | | | the hardware |
455 | | | | used to create |
456 | | | | this section. |
458 | Operating | 3 | variable | An ascii |
459 | System | | | string |
460 | | | | containing the |
461 | | | | name of the |
463 | | | | system used to |
464 | | | | create this |
467 | User | 3 | variable | An ascii |
468 | Application | | | string |
469 | | | | containing the |
470 | | | | name of the |
471 | | | | application |
472 | | | | used to create |
473 | | | | this section. |
474 +----------------+----------------+----------------+----------------+
478 The Section Header Block does not contain data but it rather
479 identifies a list of blocks (interfaces, packets) that are logically
480 correlated. This block does not contain any reference to the size of
481 the section it is currently delimiting, therefore the reader cannot
482 skip a whole section at once. In case a section must be skipped, the
483 user has to repeatedly skip all the blocks contained within it; this
484 makes the parsing of the file slower but it permits to append several
485 capture dumps at the same file.
487 3.2 Interface Description Block (mandatory)
489 The Interface Description Block is mandatory. This block is needed to
490 specify the characteristics of the network interface on which the
491 capture has been made. In order to properly associate the captured
492 data to the corresponding interface, the Interface Description Block
493 must be defined before any other block that uses it; therefore, this
494 block is usually placed immediately after the Section Header Block.
496 An Interface Description Block is valid only inside the section which
497 it belongs to. The structure of a Interface Description Block is
503 Degioanni & Risso Expires August 30, 2004 [Page 9]
505 Internet-Draft PCAP New Generation Dump File Format March 2004
509 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
510 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
511 | Interface ID | LinkType |
512 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
514 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
516 / Options (variable) /
518 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
520 Figure 4: Interface Description Block format.
522 The meaning of the fields is:
524 o Interface ID: a progressive number that identifies uniquely any
525 interface inside current section. Two Interface Description Blocks
526 can have the same Interface ID only if they are in different
527 sections of the file. The Interface ID is referenced by the packet
530 o LinkType: a value that defines the link layer type of this
533 o SnapLen: maximum number of bytes dumped from each packet. The
534 portion of each packet that exceeds this value will not be stored
537 o Options: optionally, a list of options (formatted according to the
538 rules defined in Section 4) can be present.
540 In addition to the options defined in Section 4, the following
541 options are valid within this block:
543 +----------------+----------------+----------------+----------------+
544 | Name | Code | Length | Description |
545 +----------------+----------------+----------------+----------------+
546 | if_name | 2 | Variable | Name of the |
547 | | | | device used to |
548 | | | | capture data. |
550 | if_IPv4addr | 3 | 8 | Interface |
552 | | | | address and |
555 | if_IPv6addr | 4 | 17 | Interface |
559 Degioanni & Risso Expires August 30, 2004 [Page 10]
561 Internet-Draft PCAP New Generation Dump File Format March 2004
565 | | | | address and |
566 | | | | prefix length |
567 | | | | (stored in the |
568 | | | | last byte). |
570 | if_MACaddr | 5 | 6 | Interface |
571 | | | | Hardware MAC |
572 | | | | address (48 |
575 | if_EUIaddr | 6 | 8 | Interface |
576 | | | | Hardware EUI |
577 | | | | address (64 |
581 | if_speed | 7 | 8 | Interface |
585 | if_tsaccur | 8 | 1 | Precision of |
586 | | | | timestamps. If |
588 | | | | Significant |
589 | | | | Bit is equal |
590 | | | | to zero, the |
591 | | | | remaining bits |
592 | | | | indicates the |
593 | | | | accuracy as as |
595 | | | | power of 10 |
596 | | | | (e.g. 6 means |
597 | | | | microsecond |
598 | | | | accuracy). If |
600 | | | | Significant |
601 | | | | Bit is equal |
602 | | | | to zero, the |
603 | | | | remaining bits |
604 | | | | indicates the |
605 | | | | accuracy as as |
606 | | | | negative power |
607 | | | | of 2 (e.g. 10 |
608 | | | | means 1/1024 |
609 | | | | of second). If |
610 | | | | this option is |
611 | | | | not present, a |
615 Degioanni & Risso Expires August 30, 2004 [Page 11]
617 Internet-Draft PCAP New Generation Dump File Format March 2004
620 | | | | precision of |
624 | if_tzone | 9 | 4 | Time zone for |
625 | | | | GMT support |
626 | | | | (TODO: specify |
629 | if_flags | 10 | 4 | Interface |
630 | | | | flags. (TODO: |
635 | | | | promiscuous, |
636 | | | | inbound/outbou |
637 | | | | nd, traffic |
642 | if_filter | 11 | variable | The filter |
643 | | | | (e.g. "capture |
645 | | | | traffic") used |
647 | | | | traffic. The |
648 | | | | first byte of |
650 | | | | Data keeps a |
651 | | | | code of the |
652 | | | | filter used |
653 | | | | (e.g. if this |
654 | | | | is a libpcap |
655 | | | | string, or BPF |
656 | | | | bytecode, and |
657 | | | | more). More |
658 | | | | details about |
659 | | | | this format |
661 | | | | presented in |
662 | | | | Appendix XXX |
665 | if_opersystem | 12 | variable | An ascii |
667 | | | | containing the |
671 Degioanni & Risso Expires August 30, 2004 [Page 12]
673 Internet-Draft PCAP New Generation Dump File Format March 2004
676 | | | | name of the |
678 | | | | system of the |
679 | | | | machine that |
682 | | | | This can be |
683 | | | | different from |
685 | | | | information |
686 | | | | that can be |
687 | | | | contained by |
688 | | | | the Section |
689 | | | | Header Block |
690 | | | | (Section 3.1) |
691 | | | | because the |
692 | | | | capture can |
693 | | | | have been done |
694 | | | | on a remote |
696 +----------------+----------------+----------------+----------------+
701 3.3 Packet Block (optional)
703 A Packet Block is the standard container for storing the packets
704 coming from the network. The Packet Block is optional because packets
705 can be stored either by means of this block or the Simple Packet
706 Block, which can be used to speed up dump generation. The format of a
707 packet block is shown in Figure 5.
727 Degioanni & Risso Expires August 30, 2004 [Page 13]
729 Internet-Draft PCAP New Generation Dump File Format March 2004
733 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
734 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
735 | Interface ID | Drops Count |
736 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
738 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
740 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
742 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
744 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
748 | /* variable length, byte-aligned */ |
750 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
752 / Options (variable) /
754 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
756 Figure 5: Packet Block format.
758 The Packet Block has the following fields:
760 o Interface ID: Specifies the interface this packet comes from, and
761 corresponds to the ID of one of the Interface Description Blocks
762 present in this section of the file (see Figure 4).
764 o Drops Count: a local drop counter. It specified the number of
765 packets lost (by the interface and the operating system) between
766 this packet and the preceding one. The value xFFFF (in
767 hexadecimal) is reserved for those systems in which this
768 information is not available.
770 o Timestamp (High): the most significative part of the timestamp. in
771 standard Unix format, i.e. from 1/1/1970.
773 o Timestamp (Low): the less significative part of the timestamp. The
774 way to interpret this field is specified by the 'ts_accur' option
775 (see Figure 4) of the Interface Description block referenced by
776 this packet. If the Interface Description block does not contain a
777 'ts_accur' option, then this field is expressed in microseconds.
779 o Captured Len: number of bytes captured from the packet (i.e. the
783 Degioanni & Risso Expires August 30, 2004 [Page 14]
785 Internet-Draft PCAP New Generation Dump File Format March 2004
788 length of the Packet Data field). It will be the minimum value
789 among the actual Packet Length and the snapshot length (defined in
792 o Packet Len: actual length of the packet when it was transmitted on
793 the network. Can be different from Captured Len if the user wants
794 only a snapshot of the packet.
796 o Packet Data: the data coming from the network, including
797 link-layer headers. The length of this field is Captured Len. The
798 format of the link-layer headers depends on the LinkType field
799 specified in the Interface Description Block (see Section 3.2) and
800 it is specified in Appendix XXX (TODO).
802 o Options: optionally, a list of options (formatted according to the
803 rules defined in Section 4) can be present.
806 3.4 Simple Packet Block (optional)
808 The Simple Packet Block is a lightweight container for storing the
809 packets coming from the network. Its presence is optional.
811 A Simple Packet Block is similar to a Packet Block (see Section 3.3),
812 but it is smaller, simpler to process and contains only a minimal set
813 of information. This block is preferred to the standard Packet Block
814 when performance or space occupation are critical factors, such as in
815 sustained traffic dump applications. A capture file can contain both
816 Packet Blocks and Simple Packet Blocks: for example, a capture tool
817 could switch from Packet Blocks to Simple Packet Blocks when the
818 hardware resources become critical.
820 The Simple Packet Block does not contain the Interface ID field.
821 Therefore, it must be assumed that all the Simple Packet Blocks have
822 been captured on the interface previously specified in the Interface
825 Figure 6 shows the format of the Simple Packet Block.
839 Degioanni & Risso Expires August 30, 2004 [Page 15]
841 Internet-Draft PCAP New Generation Dump File Format March 2004
845 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
846 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
848 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
852 | /* variable length, byte-aligned */ |
854 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
856 Figure 6: Simple Packet Block format.
858 The Packet Block has the following fields:
860 o Packet Len: actual length of the packet when it was transmitted on
861 the network. Can be different from captured len if the packet has
864 o Packet data: the data coming from the network, including
865 link-layers headers. The length of this field can be derived from
866 the field Block Total Length, present in the Block Header.
868 The Simple Packet Block does not contain the timestamp because this
869 is one of the most costly operations on PCs. Additionally, there are
870 applications that do not require it; e.g. an Intrusion Detection
871 System is interested in packets, not in their timestamp.
873 The Simple Packet Block is very efficient in term of disk space: a
874 snapshot of length 100 bytes requires only 16 bytes of overhead,
875 which corresponds to an efficiency of more than 86%.
877 3.5 Name Resolution Block (optional)
879 The Name Resolution Block is used to support the correlation of
880 numeric addresses (present in the captured packets) and their
881 corresponding canonical names and it is optional. Having the literal
882 names saved in the file, this prevents the need of a name resolution
883 in a delayed time, when the association between names and addresses
884 can be different from the one in use at capture time. Moreover, The
885 Name Resolution Block avoids the need of issuing a lot of DNS
886 requests every time the trace capture is opened, and allows to have
887 name resolution also when reading the capture with a machine not
888 connected to the network.
890 The format of the Name Resolution Block is shown in Figure 7.
895 Degioanni & Risso Expires August 30, 2004 [Page 16]
897 Internet-Draft PCAP New Generation Dump File Format March 2004
901 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
902 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
903 | Record Type | Record Length |
904 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
906 | /* variable length, byte-aligned */ |
907 | + + + + + + + + + + + + + + + + + + + + + + + + +
909 +-+-+-+-+-+-+-+-+ + + + + + + + + + + + + + + + + + + + + + + + +
910 . . . other records . . .
911 | Record Type == end_of_recs | Record Length == 00 |
912 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
914 / Options (variable) /
916 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
918 Figure 7: Name Resolution Block format.
920 A Name Resolution Block is a zero-terminated list of records (in the
921 TLV format), each of which contains an association between a network
922 address and a name. There are three possible types of records:
924 +----------------+----------------+----------------+----------------+
925 | Name | Code | Length | Description |
926 +----------------+----------------+----------------+----------------+
927 | end_of_recs | 0 | 0 | End of records |
929 | ip4_rec | 1 | Variable | Specifies an |
930 | | | | IPv4 address |
931 | | | | (contained in |
932 | | | | the first 4 |
934 | | | | followed by |
935 | | | | one or more |
936 | | | | zero-terminate |
938 | | | | containing the |
939 | | | | DNS entries |
943 | ip6_rec | 1 | Variable | Specifies an |
944 | | | | IPv6 address |
945 | | | | (contained in |
946 | | | | the first 16 |
951 Degioanni & Risso Expires August 30, 2004 [Page 17]
953 Internet-Draft PCAP New Generation Dump File Format March 2004
956 | | | | followed by |
957 | | | | one or more |
958 | | | | zero-terminate |
960 | | | | containing the |
961 | | | | DNS entries |
964 +----------------+----------------+----------------+----------------+
968 After the list or Name Resolution Records, optionally, a list of
969 options (formatted according to the rules defined in Section 4) can
972 A Name Resolution Block is normally placed at the beginning of the
973 file, but no assumptions can be taken about its position. Name
974 Resolution Blocks can be added in a second time by tools that process
975 the file, like network analyzers.
977 In addiction to the options defined in Section 4, the following
978 options are valid within this block:
980 +----------------+----------------+----------------+----------------+
981 | Name | Code | Length | Description |
982 +----------------+----------------+----------------+----------------+
983 | ns_dnsname | 2 | Variable | An ascii |
985 | | | | containing the |
986 | | | | name of the |
987 | | | | machine (DNS |
988 | | | | server) used |
989 | | | | to perform the |
991 | | | | resolution. |
992 +----------------+----------------+----------------+----------------+
995 3.6 Interface Statistics Block (optional)
997 The Interface Statistics Block contains the capture statistics for a
998 given interface and it is optional. The statistics are referred to
999 the interface defined in the current Section identified by the
1002 The format of the Interface Statistics Block is shown in Figure 8.
1007 Degioanni & Risso Expires August 30, 2004 [Page 18]
1009 Internet-Draft PCAP New Generation Dump File Format March 2004
1013 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1014 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1017 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1020 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1023 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1026 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1029 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1030 | Interface ID | Reserved |
1031 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1033 / Options (variable) /
1035 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1037 Figure 8: Interface Statistics Block format.
1039 The fields have the following meaning:
1041 o IfRecv: number of packets received from the interface during the
1042 capture. This number is reported as a 64 bits value, in which the
1043 most significat bits are located in the first four bytes of the
1046 o IfDrop: number of packets dropped by the interface during the
1047 capture due to lack of resources.
1049 o FilterAccept: number of packets accepeted by filter during current
1052 o OSDrop: number of packets dropped by the operating system during
1055 o UsrDelivered: number of packets delivered to the user.
1056 UsrDelivered can be different from the value 'FilterAccept -
1057 OSDropped' because some packets could still lay in the OS buffers
1058 when the capture ended.
1063 Degioanni & Risso Expires August 30, 2004 [Page 19]
1065 Internet-Draft PCAP New Generation Dump File Format March 2004
1068 o Interface ID: reference to an Interface Description Block.
1070 o Reserved: Reserved to future use.
1072 o Options: optionally, a list of options (formatted according to the
1073 rules defined in Section 4) can be present.
1075 In addiction to the options defined in Section 4, the following
1076 options are valid within this block:
1078 +----------------+----------------+----------------+----------------+
1079 | Name | Code | Length | Description |
1080 +----------------+----------------+----------------+----------------+
1081 | isb_starttime | 2 | 8 | Time in which |
1082 | | | | the capture |
1083 | | | | started; time |
1084 | | | | will be stored |
1085 | | | | in two blocks |
1086 | | | | of four bytes |
1088 | | | | containing the |
1089 | | | | timestamp in |
1090 | | | | seconds and |
1091 | | | | nanoseconds. |
1093 | isb_endtime | 3 | 8 | Time in which |
1094 | | | | the capture |
1095 | | | | started; time |
1096 | | | | will be stored |
1097 | | | | in two blocks |
1098 | | | | of four bytes |
1100 | | | | containing the |
1101 | | | | timestamp in |
1102 | | | | seconds and |
1103 | | | | nanoseconds. |
1104 +----------------+----------------+----------------+----------------+
1119 Degioanni & Risso Expires August 30, 2004 [Page 20]
1121 Internet-Draft PCAP New Generation Dump File Format March 2004
1126 Almost all blocks have the possibility to embed optional fields.
1127 Optional fields can be used to insert some information that may be
1128 useful when reading data, but that it is not really needed for packet
1129 processing. Therefore, each tool can be either read the content of
1130 the optional fields (if any), or skip them at once.
1132 Skipping all the optional fields at once is straightforward because
1133 most of the blocks have a fixed length, therefore the field Block
1134 Length (present in the General Block Structure, see Section 2.1) can
1135 be used to skip everything till the next block.
1137 Options are a list of Type - Length - Value fields, each one
1138 containing a single value:
1140 o Option Type (2 bytes): it contains the code that specifies the
1141 type of the current TLV record. Option types whose Most
1142 Significant Bit is equal to one are reserved for local use;
1143 therefore, there is no guarantee that the code used is unique
1144 among all capture files (generated by other applications). In case
1145 of vendor-specific extensions that have to be identified uniquely,
1146 vendors must request an Option Code whose MSB is equal to zero.
1148 o Option Length (2 bytes): it contains the length of the following
1149 'Option Value' field.
1151 o Option Value (variable length): it contains the value of the given
1152 option. The length of this field as been specified by the Option
1155 Options may be repeated several times (e.g. an interface that has
1156 several IP addresses associated to it). The option list is terminated
1157 by a special code which is the 'End of Option'.
1159 The format of the optional fields is shown in Figure 9.
1175 Degioanni & Risso Expires August 30, 2004 [Page 21]
1177 Internet-Draft PCAP New Generation Dump File Format March 2004
1181 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1182 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1183 | Option Code | Option Length |
1184 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1186 | /* variable length, byte-aligned */ |
1187 | + + + + + + + + + + + + + + + + + + + + + + + + +
1189 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1191 / . . . other options . . . /
1193 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1194 | Option Code == opt_endofopt | Option Length == 0 |
1195 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1197 Figure 9: Options format.
1199 The following codes can always be present in any optional field:
1201 +----------------+----------------+----------------+----------------+
1202 | Name | Code | Length | Description |
1203 +----------------+----------------+----------------+----------------+
1204 | opt_endofopt | 0 | 0 | End of |
1205 | | | | options: it is |
1207 | | | | delimit the |
1208 | | | | end of the |
1210 | | | | fields. This |
1211 | | | | block cannot |
1212 | | | | be repeated |
1213 | | | | within a given |
1217 | opt_comment | 1 | variable | Comment: it is |
1220 | | | | containing a |
1221 | | | | comment that |
1222 | | | | is associated |
1223 | | | | to the current |
1225 +----------------+----------------+----------------+----------------+
1231 Degioanni & Risso Expires August 30, 2004 [Page 22]
1233 Internet-Draft PCAP New Generation Dump File Format March 2004
1236 5. Experimental Blocks (deserved to a further investigation)
1238 5.1 Other Packet Blocks (experimental)
1240 Can some other packet blocks (besides the two described in the
1241 previous paragraphs) be useful?
1243 5.2 Compression Block (experimental)
1245 The Compression Block is optional. A file can contain an arbitrary
1246 number of these blocks. A Compression Block, as the name says, is
1247 used to store compressed data. Its format is shown in Figure 10.
1250 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1251 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1257 | /* variable length, byte-aligned */ |
1259 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1261 Figure 10: Compression Block format.
1263 The fields have the following meaning:
1265 o Compression Type: specifies the compression algorithm. Possible
1266 values for this field are 0 (uncompressed), 1 (Lempel Ziv), 2
1267 (Gzip), other?? Probably some kind of dumb and fast compression
1268 algorithm could be effective with some types of traffic (for
1269 example web), but which?
1271 o Compressed Data: data of this block. Once decompressed, it is made
1275 5.3 Encryption Block (experimental)
1277 The Encryption Block is optional. A file can contain an arbitrary
1278 number of these blocks. An Encryption Block is used to sotre
1279 encrypted data. Its format is shown in Figure 11.
1287 Degioanni & Risso Expires August 30, 2004 [Page 23]
1289 Internet-Draft PCAP New Generation Dump File Format March 2004
1293 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1294 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1300 | /* variable length, byte-aligned */ |
1302 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1304 Figure 11: Encryption Block format.
1306 The fields have the following meaning:
1308 o Compression Type: specifies the encryption algorithm. Possible
1309 values for this field are ??? NOTE: this block should probably
1310 contain other fields, depending on the encryption algorithm. To be
1313 o Encrypted Data: data of this block. Once decripted, it consists of
1317 5.4 Fixed Length Block (experimental)
1319 The Fixed Length Block is optional. A file can contain an arbitrary
1320 number of these blocks. A Fixed Length Block can be used to optimize
1321 the access to the file. Its format is shown in Figure 12. A Fixed
1322 Length Block stores records with constant size. It contains a set of
1323 Blocks (normally Packet Blocks or Simple Packet Blocks), of wihich it
1324 specifies the size. Knowing this size a priori helps to scan the file
1325 and to load some portions of it without truncating a block, and is
1326 particularly useful with cell-based networks like ATM.
1343 Degioanni & Risso Expires August 30, 2004 [Page 24]
1345 Internet-Draft PCAP New Generation Dump File Format March 2004
1349 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
1350 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1352 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
1356 | /* variable length, byte-aligned */ |
1359 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
1361 Figure 12: Fixed Length Block format.
1363 The fields have the following meaning:
1365 o Cell size: the size of the blocks contained in the data field.
1367 o Fixed Size Data: data of this block.
1370 5.5 Directory Block (experimental)
1372 If present, this block contains the following information:
1374 o number of indexed packets (N)
1376 o table with position and length of any indexed packet (N entries)
1378 A directory block must be followed by at least N packets, otherwise
1379 it must be considered invalid. It can be used to efficiently load
1380 portions of the file to memory and to support operations on memory
1381 mapped files. This block can be added by tools like network analyzers
1382 as a consequence of file processing.
1384 5.6 Traffic Statistics and Monitoring Blocks (experimental)
1386 One or more blocks could be defined to contain network statistics or
1387 traffic monitoring information. They could be use to store data
1388 collected from RMON or Netflow probes, or from other network
1391 5.7 Event/Security Block (experimental)
1393 This block could be used to store events. Events could contain
1394 generic information (for example network load over 50%, server
1395 down...) or security alerts. An event could be:
1399 Degioanni & Risso Expires August 30, 2004 [Page 25]
1401 Internet-Draft PCAP New Generation Dump File Format March 2004
1404 o skipped, if the application doesn't know how to do with it
1406 o processed independently by the packets. In other words, the
1407 applications skips the packets and processes only the alerts
1409 o processed in relation to packets: for example, a security tool
1410 could load only the packets of the file that are near a security
1411 alert; a monitorg tool could skip the packets captured while the
1455 Degioanni & Risso Expires August 30, 2004 [Page 26]
1457 Internet-Draft PCAP New Generation Dump File Format March 2004
1462 The file format proposed in this document should be very versatile
1463 and satisfy a wide range of applications. In the simplest case, it
1464 can contain a raw dump of the network data, made of a series of
1465 Simple Packet Blocks. In the most complex case, it can be used as a
1466 repository for heterogeneous information. In every case, the file
1467 remains easy to parse and an application can always skip the data it
1468 is not interested in; at the same time, different applications can
1469 share the file, and each of them can benfit of the information
1470 produced by the others. Two or more files can be concatenated
1471 obtaining another valid file.
1511 Degioanni & Risso Expires August 30, 2004 [Page 27]
1513 Internet-Draft PCAP New Generation Dump File Format March 2004
1516 7. Most important open issues
1518 o Data, in the file, must be byte or word aligned? Currently, the
1519 structure of this document is not consistent with respect to this
1567 Degioanni & Risso Expires August 30, 2004 [Page 28]
1569 Internet-Draft PCAP New Generation Dump File Format March 2004
1572 Intellectual Property Statement
1574 The IETF takes no position regarding the validity or scope of any
1575 intellectual property or other rights that might be claimed to
1576 pertain to the implementation or use of the technology described in
1577 this document or the extent to which any license under such rights
1578 might or might not be available; neither does it represent that it
1579 has made any effort to identify any such rights. Information on the
1580 IETF's procedures with respect to rights in standards-track and
1581 standards-related documentation can be found in BCP-11. Copies of
1582 claims of rights made available for publication and any assurances of
1583 licenses to be made available, or the result of an attempt made to
1584 obtain a general license or permission for the use of such
1585 proprietary rights by implementors or users of this specification can
1586 be obtained from the IETF Secretariat.
1588 The IETF invites any interested party to bring to its attention any
1589 copyrights, patents or patent applications, or other proprietary
1590 rights which may cover technology that may be required to practice
1591 this standard. Please address the information to the IETF Executive
1595 Full Copyright Statement
1597 Copyright (C) The Internet Society (2004). All Rights Reserved.
1599 This document and translations of it may be copied and furnished to
1600 others, and derivative works that comment on or otherwise explain it
1601 or assist in its implementation may be prepared, copied, published
1602 and distributed, in whole or in part, without restriction of any
1603 kind, provided that the above copyright notice and this paragraph are
1604 included on all such copies and derivative works. However, this
1605 document itself may not be modified in any way, such as by removing
1606 the copyright notice or references to the Internet Society or other
1607 Internet organizations, except as needed for the purpose of
1608 developing Internet standards in which case the procedures for
1609 copyrights defined in the Internet Standards process must be
1610 followed, or as required to translate it into languages other than
1613 The limited permissions granted above are perpetual and will not be
1614 revoked by the Internet Society or its successors or assignees.
1616 This document and the information contained herein is provided on an
1617 "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
1618 TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
1619 BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
1623 Degioanni & Risso Expires August 30, 2004 [Page 29]
1625 Internet-Draft PCAP New Generation Dump File Format March 2004
1628 HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
1629 MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
1634 Funding for the RFC Editor function is currently provided by the
1679 Degioanni & Risso Expires August 30, 2004 [Page 30]