1 This is recutils.info, produced by makeinfo version 6.3 from
4 This manual is for GNU recutils (version 1.8, 3 January 2019).
6 Copyright (C) 2009-2019 Jose E. Marchesi
8 Copyright (C) 1994-2014 Free Software Foundation, Inc.
10 Permission is granted to copy, distribute and/or modify this
11 document under the terms of the GNU Free Documentation License,
12 Version 1.3 or any later version published by the Free Software
13 Foundation; with no Invariant Sections, no Front-Cover Texts, and
14 no Back-Cover Texts. A copy of the license is included in the
15 section entitled "GNU Free Documentation License".
16 INFO-DIR-SECTION Database
18 * recutils: (recutils). The GNU Recutils manual.
21 INFO-DIR-SECTION Individual utilities
23 * recinf: (recutils)Invoking recinf. Get info about recfiles.
24 * recsel: (recutils)Invoking recsel. Read records.
25 * recins: (recutils)Invoking recins. Insert records.
26 * recdel: (recutils)Invoking recdel. Delete records.
27 * recset: (recutils)Invoking recset. Manage fields.
28 * recfix: (recutils)Invoking recfix. Fix recfiles.
29 * csv2rec: (recutils)Invoking csv2rec. CSV to recfiles.
30 * rec2csv: (recutils)Invoking rec2csv. Recfiles to CSV.
31 * mdb2rec: (recutils)Invoking mdb2rec. MDB to recfiles.
35 File: recutils.info, Node: Top, Next: Introduction, Up: (dir)
40 This manual documents version 1.8 of the GNU recutils.
42 This manual is for GNU recutils (version 1.8, 3 January 2019).
44 Copyright (C) 2009-2019 Jose E. Marchesi
46 Copyright (C) 1994-2014 Free Software Foundation, Inc.
48 Permission is granted to copy, distribute and/or modify this
49 document under the terms of the GNU Free Documentation License,
50 Version 1.3 or any later version published by the Free Software
51 Foundation; with no Invariant Sections, no Front-Cover Texts, and
52 no Back-Cover Texts. A copy of the license is included in the
53 section entitled "GNU Free Documentation License".
58 * Introduction:: Introducing recutils.
59 * The Rec Format:: Writing recfiles.
62 * Querying Recfiles:: Extracting data from recfiles.
63 * Editing Records:: Inserting and deleting records.
64 * Editing Fields:: Inserting, modifying and deleting fields.
67 * Field Types:: Restrictions on the values of fields.
68 * Constraints on Record Sets:: Requiring or forbidding specific fields.
69 * Checking Recfiles:: Making sure the data is ok.
72 * Remote Descriptors:: Implementing distributed databases.
73 * Grouping and Aggregates:: Statistics.
74 * Queries which Join Records:: Crossing record of different types.
75 * Auto-Generated Fields:: Counters and time-stamps.
76 * Encryption:: Storing sensitive information.
77 * Generating Reports:: Formatted output with templates.
78 * Interoperability:: Importing and exporting to other formats.
79 * Bash Builtins:: Boosting the recutils in the shell.
82 * Invoking the Utilities:: Exhaustive list of command line arguments.
83 * Regular Expressions:: Flavor of regexps supported in recutils.
84 * Date input formats:: Specifying dates and times.
86 * GNU Free Documentation License:: Distribution terms for this document.
91 -- The Detailed Node Listing --
92 ----------------------
94 Here are some other nodes which are really subnodes of the ones
95 already listed, mentioned here so you can get to them in one step:
99 * Purpose:: Why recutils.
100 * A Little Example:: Recutils in action.
104 * Fields:: The key-value pairs which comprise the data.
105 * Records:: The main entities of a recfile.
106 * Comments:: Information for humans' benefit only.
107 * Record Descriptors:: Describing different types of records.
111 * Simple Selections:: Introducing 'recsel'.
112 * Selecting by Type:: Get the records of some given type.
113 * Selecting by Position:: Get the record occupying some position.
114 * Random Records:: Get a set of random records.
115 * Selection Expressions:: Get the records satisfying some expression.
116 * Field Expressions:: Selecting a subset of fields.
117 * Sorted Output:: Get the records in a given order.
121 * Inserting Records:: Inserting data into recfiles.
122 * Deleting Records:: Removing entries.
123 * Sorting Records:: Physical reordering of records.
127 * Setting Fields:: Editing field values.
128 * Adding Fields:: Adding new fields to records.
129 * Deleting Fields:: Removing or commenting-out fields.
133 * Declaring Types:: Declaration of types in record descriptors.
134 * Types and Fields:: Associating fields with types.
135 * Scalar Field Types:: Numbers and ranges.
136 * String Field Types:: Lines, limited strings and regular expressions.
137 * Enumerated Field Types:: Enumerations and boolean values.
138 * Date and Time Types:: Dates and times.
139 * Other Field Types:: Emails, fields, UUIDs, ...
141 Constraints on Record Sets
143 * Mandatory Fields:: Requiring the presence of fields.
144 * Prohibited Fields:: Forbidding the presence of fields.
145 * Allowed Fields:: Restricting the presence of fields.
146 * Keys and Unique Fields:: Fields characterizing records.
147 * Size Constraints:: Limiting the size of a record set.
148 * Arbitrary Constraints:: Constraints records must comply with.
152 * Syntactical Errors:: Fixing structure errors in recfiles.
153 * Semantic Errors:: Fixing semantic errors in recfiles.
157 Grouping and Aggregates
159 * Grouping Records:: Combining records by fields.
160 * Aggregate Functions:: Statistics and more.
164 * Foreign Keys:: Referring records from another records.
165 * Joining Records:: Performing cross-joins.
167 Auto-Generated Fields
169 * Counters:: Generating incremental Ids.
170 * Unique Identifiers:: Generating universally unique Ids.
171 * Time-Stamps:: Tracking the creation of records.
175 * Confidential Fields:: Declaring fields as sensitive data.
176 * Encrypting Files:: Encrypt confidential fields.
177 * Decrypting Data:: Reading encrypted fields.
181 * Templates:: Formatted output.
185 * CSV Files:: Converting recfiles to/from csv files.
186 * Importing MDB Files:: Importing MS Access Databases.
190 * readrec:: Exporting the contents of records to the shell.
192 Invoking the Utilities
194 * Invoking recinf:: Printing information about rec files.
195 * Invoking recsel:: Selecting records.
196 * Invoking recins:: Inserting records.
197 * Invoking recdel:: Deleting records.
198 * Invoking recset:: Managing fields.
199 * Invoking recfix:: Fixing broken rec files, and diagnostics.
200 * Invoking recfmt:: Formatting records using templates.
201 * Invoking csv2rec:: Converting csv data into rec data.
202 * Invoking rec2csv:: Converting rec data into csv data.
203 * Invoking mdb2rec:: Converting mdb files into rec files.
207 File: recutils.info, Node: Introduction, Next: The Rec Format, Prev: Top, Up: Top
214 * Purpose:: Why recutils.
215 * A Little Example:: Recutils in action.
218 File: recutils.info, Node: Purpose, Next: A Little Example, Up: Introduction
223 GNU recutils is a set of tools and libraries to access human-editable,
224 text-based databases called _recfiles_. The data is stored as a
225 sequence of records, each record containing an arbitrary number of named
226 fields. Advanced capabilities usually found in other data storage
227 systems are supported: data types, data integrity (keys, mandatory
228 fields, etc.) as well as the ability of records to refer to other
229 records (sort of foreign keys). Despite its simplicity, recfiles can be
230 used to store medium-sized databases.
232 So, yet another data storage system? The mere existence of this
233 package deserves an explanation. There is a rich set of already
234 available free data storage systems, covering a broad range of
235 requirements. Big systems having complex data storage requirements will
236 probably make use of some full-fledged relational system such as MySQL
237 or PostgreSQL. Less demanding applications, or applications with
238 special deployment requirements, may find it more convenient to use a
239 simpler system such as SQLite, where the data is stored in a single
240 binary file. XML files are often used to store configuration settings
241 for programs, and to encode data for transmission through networks.
243 So it looks like all the needs are covered by the existing solutions
244 ... but consider the following characteristics of the data storage
245 systems mentioned in the previous paragraph:
247 - The stored data is not directly human readable.
248 - The stored data is definitely not directly writable by humans.
249 - They are program dependent.
250 - They are not easily managed by version control systems.
252 Regarding the first point (human readability), while it is clearly
253 true for the binary files, some may argue XML files are indeed human
254 readable... well... '<bar><foo tag="val">try</foo> to r&iamp;ead
255 <p>this</p></bar>'. YAML (1) is an example of a hierarchical data
256 storage format which is much more readable than XML. The problem with
257 YAML is that it was designed as a "data serialization language" and thus
258 to map the data constructs usually found in programming languages. That
259 makes it too complex for the simple task of storing plain lists of
262 Recfiles are human-readable, human-writable and still easy to parse
263 and to manipulate automatically. Obviously they are not suitable for
264 any task (for example, it can be difficult to manage hierarchies in
265 recfiles) and performance is somewhat sacrificed in favor of
266 readability. But they are quite handy to store small to medium simple
269 The GNU recutils suite comprises:
271 - This Texinfo manual, describing the Rec format and the accompanying
273 - A C library (librec) that provides a rich set of functions to
275 - A set utilities that can be used in shell scripts and in the
276 command line to operate on rec files.
277 - An emacs mode, 'rec-mode'.
279 ---------- Footnotes ----------
281 (1) Yet Another Markup Language
284 File: recutils.info, Node: A Little Example, Prev: Purpose, Up: Introduction
289 Everyone loves to grow a nice book collection at home. Unfortunately,
290 in most cases the management of our private books gets uncontrolled:
291 some books get lost, some of them may be loaned to some friend, there
292 are some duplicated (or even triplicated!) titles because we forgot
293 about the existence of the previous copy, and many more details.
295 In order to improve the management of our little book collection we
296 could make use of a complex data storage system such as a relational
297 database. The problem with that approach, as explained in the previous
298 section, is that the tool is too complicated for the simple task: we do
299 not need the full power of a relational database system to maintain a
300 simple collection of books.
302 With GNU recutils it is possible to maintain such a little database
303 in a text file. Let's call it 'books.rec'. The following table resumes
304 the information items that we want to store for each title, along with
305 some common-sense restrictions.
307 - Every book has a title, even if it is "No Title".
308 - A book can have several titles.
309 - A book can have more than one author.
310 - For some books the author is not known.
311 - Sometimes we don't care about who the author of a book is.
312 - We usually store our books at home.
313 - Sometimes we loan books to friends.
314 - On occasions we lose track of the physical location of a book. Did
315 we loan it to anyone? Was it lost in the last move? Is it in some
316 hidden place at home?
318 The contents of the rec file follows:
324 %type: Location enum loaned home unknown
326 + A book in my personal collection.
328 Title: GNU Emacs Manual
329 Author: Richard M. Stallman
333 Title: The Colour of Magic
334 Author: Terry Pratchett
341 Title: chapters.gnu.org administration guide
342 Author: Nacho Gonzalez
343 Author: Jose E. Marchesi
346 Title: Yeelong User Manual
351 Simple. The file contains a set of records separated by blank lines.
352 Each record comprises a set of fields with a name and a value.
354 The GNU recutils can then be used to access the contents of the file.
355 For example, we could get a list of the names of loaned books by
356 invoking 'recsel' in the following way:
358 $ recsel -e "Location = 'loaned'" -P Title books.rec
362 File: recutils.info, Node: The Rec Format, Next: Querying Recfiles, Prev: Introduction, Up: Top
367 A recfile is nothing but a text file which conforms to a few simple
368 rules. This chapter shows you how, by observing these rules, recfiles
369 of arbitrary complexity can be written.
373 * Fields:: The key-value pairs which comprise the data.
374 * Records:: The main entities of a recfile.
375 * Comments:: Information for humans' benefit only.
376 * Record Descriptors:: Describing different types of records.
379 File: recutils.info, Node: Fields, Next: Records, Up: The Rec Format
384 A "field" is the written form of an association between a label and a
385 value. For example, if we wanted to associate the label 'Name' with the
386 value 'Ada Lovelace' we would write:
390 The separator between the field name and the field value is a colon
391 followed by a blank character (space and tabs, but not newlines). The
392 name of the field shall begin in the first column of the line.
394 A "field name" is a sequence of alphanumeric characters plus
395 underscores ('_'), starting with a letter or the character '%'. The
396 regular expression denoting a field name is:
398 [a-zA-Z%][a-zA-Z0-9_]*
400 Field names are case-sensitive. 'Foo' and 'foo' are different field
403 The following list contains valid field names (the final colon is not
412 The "value of a field" is a sequence of characters terminated by a
413 single newline character ('\n').
415 Sometimes a value is too long to fit in the usual width of terminals
416 and screens. In that case, depending on the specific tool used to
417 access the file, the readability of the data would not be that good. It
418 is therefore possible to physically split a logical line by escaping a
419 newline with a backslash character, as in:
421 LongLine: This is a quite long value \
422 comprising a single unique logical line \
423 split in several physical lines.
425 The sequence '\n' (newline) '+' (PLUS) and an optional '_' (SPACE) is
426 interpreted as a newline when found in a field value. For example, the
427 C string '"bar1\nbar2\n bar3"' would be encoded in the following way in
435 File: recutils.info, Node: Records, Next: Comments, Prev: Fields, Up: The Rec Format
440 A "record" is a group of one or more fields written one after the other:
446 It is possible for several fields in a record to share the same name
447 or/and the field value. The following is a valid record containing
451 Email: john.smith@foomail.com
452 Email: john@smith.name
454 The "size of a record" is defined as the number of fields that it
455 contains. A record cannot be empty, so the minimum size for a record is
456 1. The maximum number of fields for a record is only limited by the
457 available physical resources. The size of the previous record is 3.
459 Records are separated by one or more blank lines. For instance, the
460 following example shows a file named 'personalities.rec' featuring three
466 Name: Peter the Great
473 File: recutils.info, Node: Comments, Next: Record Descriptors, Prev: Records, Up: The Rec Format
478 Any line having an '#' (ASCII 0x23) character in the first column is a
481 Comments may be used to insert information that is not part of the
482 database but useful in other ways. They are completely ignored by
483 processing tools and can only be seen by looking at the recfile itself.
485 It is also quite convenient to comment-out information from the
486 recfile without having to remove it in a definitive way: you may want to
487 recover the data into the database later! Comment lines can be used to
488 comment-out both full registers and single fields:
490 Name: Jose E. Marchesi
491 # Occupation: Software Engineer
492 # Severe lack of brain capacity
493 # Fired on 02/01/2009 (without compensation)
494 Occupation: Unoccupied
496 Comments are also useful for headers, footers, comment blocks and all
503 # This file contains the Bugs database of GNU recutils.
511 Unlike some file formats, comments in recfiles must be complete
512 lines. You cannot start a comment in the middle of a line. For
513 example, in the following record, the '#' does _not_ start a comment:
514 Name: Peter the Great # Russian Tsar
518 File: recutils.info, Node: Record Descriptors, Prev: Comments, Up: The Rec Format
520 2.4 Record Descriptors
521 ======================
523 Certain properties of a set of records can be specified by preceding
524 them with a "record descriptor". A record descriptor is itself a
525 record, and uses fields with some predefined names to store properties.
529 * Record Sets:: Defining different types of records.
530 * Naming Record Types:: Some conventions on naming record sets.
531 * Documenting Records:: Documenting your record sets.
532 * Record Sets Properties:: Introducing the special fields.
535 File: recutils.info, Node: Record Sets, Next: Naming Record Types, Up: Record Descriptors
540 The most basic property that can be specified for a set of records is
541 their "type". The special field name '%rec' is used for that purpose:
551 The records following the descriptors are then identified as having
552 its type. So in the example above we would say there are two records of
553 type "Entry". Or in a more colloquial way we would say there are two
554 "Entries" in the database.
556 The effect of a record descriptor ends when another descriptor is
557 found in the stream of records. This allows you to store different
558 kinds of records in the same database. For example, suppose you are
559 maintaining a depot. You will need to keep track of both what items are
560 available and when they are sold or restocked.
562 The following example shows the usage of two record descriptors to
563 store both kind of records: articles and stock.
583 The collection of records having same types in recfiles are known as
584 "record sets" in recutils jargon. In the example above two record sets
585 are defined: one containing articles and the other containing stock
588 Nothing prevents having empty record sets in databases. This is in
589 fact usually the case when a new recfile is written but no data exists
590 yet. In our depot example we could write a first version of the
591 database containing just the record descriptors:
597 Special records are not required, and many recfiles do not have them.
598 This is because all the records contained in the file are of the same
599 type, and their nature can usually be inferred from both the file name
600 and their contents. For example, 'contacts.rec' could simply contain
601 records representing contacts without an explicit '%rec: Contact' record
602 descriptor. In this case we say that the type of the anonymous records
603 stored in the file is the "default record type".
605 Another possible situation, although not usual, is to have a recfile
606 containing both non-typed (default) and typed record types:
622 In this case the records preceding the movements are of the "default"
623 type, whereas the records following the record descriptor are of type
624 'Movement'. Even though it is supported by the format and the
625 utilities, it is generally not recommended to mix non-typed and typed
626 records in a recfile.
629 File: recutils.info, Node: Naming Record Types, Next: Documenting Records, Prev: Record Sets, Up: Record Descriptors
631 2.4.2 Naming Record Types
632 -------------------------
634 It is up to you how to name your record sets. Any string comprising
635 only alphanumeric characters or underscores, and that starts with a
636 letter will be a legal name. However, it is recommended to use the
637 singular form of a noun in order to describe the "type" of the records
638 in the records set. Examples are 'Article', 'Contributor', 'Employee'
641 The used noun should be specific enough in order to characterize the
642 property of the records which matters. For example, in a contributor's
643 database it would be better to have a record set named 'Contributor'
646 The reason of using singular nouns instead of their plural forms is
647 that it works better with the utilities: it is more natural to read
648 'recsel -t Contributor' ('-t' is for "type") than 'recsel -t
652 File: recutils.info, Node: Documenting Records, Next: Record Sets Properties, Prev: Naming Record Types, Up: Record Descriptors
654 2.4.3 Documenting Records
655 -------------------------
657 As well as a name, it is a good idea to provide a description of the
658 record set. This is sometimes called the record set's "documentation"
659 and is specified using the '%doc' field.
661 Whereas the name is usually short and can contain only alphanumeric
662 characters and underscores, no such restriction applies to the
663 documentation. The documentation is typically more verbose than the
664 name provided by the '%rec' field and may contain arbitrary characters
665 such as punctuation and parentheses. It is somewhat similar to a
666 comment (*note Comments::), but it can be managed more easily in a
667 programmatic way. Unlike a comment, the '%doc' field is recognized by
668 tools such as 'recinf' (*note Invoking recinf::) which processes record
669 descriptors. For example, you might have two record sets with '%rec'
670 and '%doc' fields as follows:
673 %doc: Family, friends and acquaintances (other than business).
683 %doc: Colleagues and other business contacts
688 Name: Genevieve Curie
692 File: recutils.info, Node: Record Sets Properties, Prev: Documenting Records, Up: Record Descriptors
694 2.4.4 Record Sets Properties
695 ----------------------------
697 Besides determining the type of record that follows in the stream,
698 record descriptors can be used to describe other properties of those
699 records. This can be done by using "special fields", which have special
700 names from a predefined set. Consider for example the following
701 database, where record descriptors are used to specify a (optional)
702 numeric 'Id' and a mandatory 'Title' field:
709 Title: Notebook (big)
714 Note that the names of special fields always start with the character
715 '%'. Also note that it is also possible to use non-special fields in a
716 record descriptor, but such fields will have no effect on the described
719 Every record set must contain one, and only one, field named '%rec'.
720 It is not mandated that that field must occupy the first position in the
721 record. However, it is considered a good style to place it as the first
722 field in the record set, in order for the casual reader to easily
723 identify the type of the records.
725 The following list briefly describes the special fields defined in
726 the recutils format, along with references to the sections of this
727 manual describing their usage in depth.
730 Naming record types. Also, they allow using external and remote
731 descriptors. *Note Remote Descriptors::.
732 '%mandatory, %allowed and %prohibit'
733 Requiring or forbidding specific fields. *Note Mandatory Fields::.
734 *Note Prohibited Fields::. *Note Allowed Fields::.
736 Working with keys. *Note Keys and Unique Fields::.
738 Documenting your database. *Note Documenting Records::.
740 Field types. *Note Field Types::.
742 Auto-counters and time-stamps. *Note Auto-Generated Fields::.
744 Keeping your record sets sorted. *Note Sorted Output::.
746 Restricting the size of your database. *Note Size Constraints::.
748 Enforcing arbitrary constraints. *Note Arbitrary Constraints::.
750 Storing confidential information. *Note Encryption::.
753 File: recutils.info, Node: Querying Recfiles, Next: Editing Records, Prev: The Rec Format, Up: Top
758 Since recfiles are always human readable, you could lookup data simply
759 by opening an editor and searching for the desired information. Or you
760 could use a standard tool such as 'grep' to extract strings matching a
761 pattern. However, recutils provides a more powerful and flexible way to
762 lookup data. The following sections explore how the recutils can be
763 used in order to extract data from recfiles, from very basic and simple
764 queries to quite complex examples.
768 * Simple Selections:: Introducing 'recsel'.
769 * Selecting by Type:: Get the records of some given type.
770 * Selecting by Position:: Get the record occupying some position.
771 * Random Records:: Get a set of random records.
772 * Selection Expressions:: Get the records satisfying some expression.
773 * Field Expressions:: Selecting a subset of fields.
774 * Sorted Output:: Get the records in a given order.
777 File: recutils.info, Node: Simple Selections, Next: Selecting by Type, Up: Querying Recfiles
779 3.1 Simple Selections
780 =====================
782 'recsel' is an utility whose primary purpose is to select records from a
783 recfile and print them on standard output. Consider the following
784 example record set, which we shall assume is saved in a recfile called
787 # This database contains a list of both real and fictional people
788 # along with their age.
793 Name: Peter the Great
805 If we invoke 'recsel acquaintances.rec' we will get a list of all the
806 records stored in the file in the terminal:
808 $ recsel acquaintances.rec
812 Name: Peter the Great
821 Note that the commented out parts of the file, in this case the
822 explanatory header and the record corresponding to Matusalem, are not
823 part of the output produced by 'recsel'. This is because 'recsel' is
824 concerned only with the data.
826 'recsel' will also "pack" the records so any extra empty lines that
827 may be between records are not echoed in the output:
829 *acquaintances.rec:* $ recsel acquaintances.rec
830 Name: Peter the Great
831 Name: Peter the Great Age: 53
834 # Note the extra empty lines. Age: 10
840 It is common to store data gathered in several recfiles. For example,
841 we could have a 'contacts.rec' file containing general contact records,
842 and also a 'work-contacts.rec' file containing business contacts:
844 *contacts.rec:* *work-contacts.rec:*
846 Name: Granny Name: Yoyodyne Corp.
847 Phone: +12 23456677 Email: sales@yoyod.com
850 Phone: +12 58999222 Name: Robert Harris
851 Email: robert.harris@yoyod.com
852 Note: Sales Department.
854 Both files can be passed to 'recsel' in the command line. In that
855 case 'recsel' will simply process them and output their records in the
856 same order they were specified:
858 $ recsel contacts.rec work-contacts.rec
866 Email: sales@yoyod.com
870 Email: robert.harris@yoyod.com
871 Note: Sales Department.
873 As mentioned above, the output follows the ordering on the command line,
874 so 'recsel work-contacts.rec contacts.rec' would output the records of
875 'work-contacts.rec' first and then the ones from 'contacts.rec'.
877 Note however that 'recsel' will merge records from several files
878 specified in the command line only if they are anonyomuse. If the
879 contacts in our files were typed:
881 *contacts.rec:* *work-contacts.rec:*
883 %rec: Contact %rec: Contact
885 Name: Granny Name: Yoyodyne Corp.
886 Phone: +12 23456677 Email: sales@yoyod.com
889 Phone: +12 58999222 Name: Robert Harris
890 Email: robert.harris@yoyod.com
891 Note: Sales Department.
893 Then we would get the following error message:
895 $ recsel contacts.rec work-contacts.rec
896 recsel: error: duplicated record set 'Contact' from work-contacts.rec.
899 File: recutils.info, Node: Selecting by Type, Next: Selecting by Position, Prev: Simple Selections, Up: Querying Recfiles
901 3.2 Selecting by Type
902 =====================
904 As we saw in the section discussing record descriptors, it is possible
905 to have several different types of records in a single recfile.
906 Consider for example a 'gnu.rec' file containing information about
907 maintainers and packages in the GNU Project:
911 Name: Jose E. Marchesi
912 Email: jemarch@gnu.org
915 Email: positron@gnu.org
920 LastRelease: 12 February 2014
923 LastRelease: 10 March 2013
925 If 'recsel' is invoked in that file it will complain:
928 recsel: error: several record types found. Please use -t to specify one.
930 This is because 'recsel' does not know which records to output: the
931 maintainers or the packages. This can be resolved by using the '-t'
934 $ recsel -t Package gnu.rec
936 LastRelease: 12 February 2014
939 LastRelease: 10 March 2013
941 By default 'recsel' never outputs record descriptors. This is because
942 most of the time the user is only interested in the data. However, with
943 the '-d' command line option, the record descriptor of the selected type
944 is printed preceding the data records:
946 $ recsel -d -t Maintainer gnu.rec
949 Name: Jose E. Marchesi
950 Email: jemarch@gnu.org
953 Email: positron@gnu.org
955 Note that at the moment it is not possible to select non-typed (default)
956 records when other record sets are stored in the same file. This is one
957 of the reasons why mixing non-typed records and typed records in a
958 single recfile is not recommended.
960 Note also that if a nonexistent record type is specified in '-t' then
961 'recsel' does nothing.
964 File: recutils.info, Node: Selecting by Position, Next: Random Records, Prev: Selecting by Type, Up: Querying Recfiles
966 3.3 Selecting by Position
967 =========================
969 As was explained in the previous sections, 'recsel' outputs all the
970 records of some record set. The records are echoed in the same order
971 they are written in the recfile. However, often it is desirable to
972 select a subset of the records, determined by the position they occupy
975 The '-n' command line option to 'recsel' supports doing this in a
976 natural way. This is how we would retrieve the first contact listed in
977 a contacts database using 'recsel':
979 $ recsel -n 0 contacts.rec
983 Note that the index is zero-based. If we want to retrieve more records
984 we can specify several indexes to '-n' separated by commas. If a given
985 index is too big, it is simply ignored:
987 $ recsel -n 0,1,999 contacts.rec
994 With '-n', the order in which the records are echoed does not depend on
995 the order of the indexes passed to '-n'. For example, the output of
996 'recsel -n 0,1' will be identical to the output of 'recsel -n 1,0'.
998 Ranges of indexes can also be used to select a subset of the records.
999 For example, the following call would also select the first three
1000 contacts of the database:
1002 $ recsel -n 0-2 contacts.rec
1012 It is possible to mix single indexes and index ranges in the same call.
1013 For example, 'recsel -n 0,5-6' would select the first, sixth and seventh
1017 File: recutils.info, Node: Random Records, Next: Selection Expressions, Prev: Selecting by Position, Up: Querying Recfiles
1022 Consider a database in which each record is a cooking recipe. It is
1023 always difficult to decide what to cook each day, so it would be nice if
1024 we could ask 'recsel' to pick up a random recipe. This can be achieved
1025 using the '-m' ('--random') command line option of 'recsel':
1027 $ recsel -m 1 recipes.rec
1028 Title: Curry chicken
1029 Ingredient: A whole chicken
1033 If we need two recipes, because we will be cooking at both lunch and
1034 dinner, we can pass a different number to '-m':
1036 $ recsel -m 2 recipes.rec
1037 Title: Fabada Asturiana
1038 Ingredient: 300 gr of fabes.
1040 Ingredient: Morcilla
1043 Title: Pasta with ragu
1044 Ingredient: 500 gr of spaghetti.
1045 Ingredient: 2 tomatoes.
1046 Ingredient: Minced meat.
1049 The algorithm used to implement '-m' guarantees that you will never get
1050 multiple instances of the same record. This means that if a record set
1051 has N records and you ask for N random records, you will get all the
1052 records in a random order.
1055 File: recutils.info, Node: Selection Expressions, Next: Field Expressions, Prev: Random Records, Up: Querying Recfiles
1057 3.5 Selection Expressions
1058 =========================
1060 "Selection expressions", also known as "sexes" in recutils jargon, are
1061 infix expressions that can be applied to a record. A "sex" is a
1062 predicate which selects a subset of records within a recfile. They can
1063 be simple expressions involving just one operator and a pair of
1064 operands, or complex compound expressions with parenthetical
1065 sub-expressions and many operators and operands. One of their most
1066 common uses is to examine records matching a particular set of
1071 * Selecting by predicate:: Selecting records which satisfy conditions.
1072 * SEX Operands:: Literal values, fields and sub-expressions.
1073 * SEX Operators:: Arithmetic, logical and other operators.
1074 * SEX Evaluation:: Selection expressions are like generators.
1077 File: recutils.info, Node: Selecting by predicate, Next: SEX Operands, Up: Selection Expressions
1079 3.5.1 Selecting by predicate
1080 ----------------------------
1082 Consider the example recfile 'acquaintances.rec' introduced earlier. It
1083 contains names of people along with their respective ages. Suppose we
1084 want to get a list of the names of all the children. It would not be
1085 easy to do this using 'grep'. Neither would it, for any reasonably
1086 large recfile, be feasible to search manually for the children.
1087 Fortunately the 'recsel' command provides an easy way to do such a
1089 $ recsel -e "Age < 18" -P Name acquaintances.rec
1093 Let us look at each of the arguments to 'recsel' in turn. Firstly we
1094 have '-e' which tells 'recsel' to lookup records matching the expression
1095 'Age < 18' -- in other words all those people whose ages are less than
1096 18. This is an example of a "selection expression". In this case it is
1097 a simple test, but it can be as complex as needed.
1099 Next, there is '-P' which tells 'recsel' to print out the value of
1100 the 'Name' field -- because we want just the name, not the entire
1101 record. The final argument is the name of the file from whence the
1102 records are to come: 'acquaintances.rec'.
1104 Rather than explicitly storing ages in the recfile, a more realistic
1105 example might have the date of birth instead (otherwise it would be
1106 necessary to update the people's ages in the recfile on every birthday).
1113 Email: alf@example.com
1115 Name: Bertram Worcester
1117 Email: bert@example.com
1119 Name: Charles Spencer
1121 Email: charlie@example.com
1125 Email: dirk@example.com
1129 Email: ernie@example.com
1131 Now we can achieve a similar result as before, by looking up the names
1132 of all those people who were born after a particular date:
1133 $ recfix acquaintances.rec
1134 $ recsel -e "Dob >> '31 July 1994'" -p Name acquaintances.rec
1136 Name: Charles Spencer
1138 The '>>' operator means "later than", and is used here to select a date
1139 of birth after 31st July 1994. Note also that this example uses a lower
1140 case '-p' whereas the preceding example used the upper case '-P'. The
1141 difference is that '-p' prints the field name and field value, whereas
1142 '-P' prints just the value.
1144 'recsel' accepts more than one '-e' argument, each introducing a
1145 selection expression, in which case the records which satisfy all
1146 expressions are selected. You can provide more than one field label to
1147 '-P' or '-p' in order to select additional fields to be displayed. For
1148 example, if you wanted to send an email to all children 14 to 18 years
1149 of age, and today's date were 1st August 2012, then you could use the
1150 following command to get the name and email address of all such
1153 $ recfix acquaintances.rec
1154 $ recsel -e "Dob >> '31 July 1994' && Dob << '01 August 1998'" \
1155 -p Name,Email acquaintances.rec
1156 Name: Charles Spencer
1157 Email: charlie@example.com
1159 As you can see, there is only one such child in our record set.
1161 Note that the example command shown above contains both double quotes
1162 '"' and single quotes '''. The double quotes are interpreted by the
1163 shell (e.g. 'bash') and the single quotes are interpreted by 'recsel',
1164 defining a string. (And the backslash is interpreted by the shell, the
1165 usual continuation character so that this manual doesn't have a too-long
1169 File: recutils.info, Node: SEX Operands, Next: SEX Operators, Prev: Selecting by predicate, Up: Selection Expressions
1174 The supported operands are: numbers, strings, field names and
1175 parenthesized expressions.
1177 3.5.2.1 Numeric Literals
1178 ........................
1180 The supported numeric literals are integer numbers and real numbers.
1181 The usual sign character '-' is used to denote negative values. Integer
1182 values can be denoted in base 10, base 16 using the '0x' prefix, and
1183 base 8 using the '0' prefix. Examples are:
1195 3.5.2.2 String Literals
1196 .......................
1198 String values are delimited by either the ''' character or the '"'
1199 character. Whichever delimiter is used, the delimiter closing the
1200 literal must be the same as the delimiter used to open it.
1202 Newlines and tabs can be part of a string literal.
1207 'The following example is the empty string.'
1210 The ''' and '"' characters can be part of a string if they are
1211 escaped with a backslash, as in:
1213 'This string contains an apostrophe: \'.'
1214 "This one a double quote: \"."
1216 3.5.2.3 Field Values
1217 ....................
1219 The value of a field value can be included in a selection expression by
1220 writing its name. The field name is replaced by a string containing the
1221 field value, to handle the possibility of records with more than one
1222 field by that name. Examples:
1228 It is possible to use the role part of a field if it is not empty.
1229 So, for example, if we are searching for the issues opened by 'John
1230 Smith' in a database of issues we could write:
1232 $ recsel -e "OpenedBy = 'John Smith'"
1234 instead of using a full field name:
1236 $ recsel -e "Hacker:Name:OpenedBy = 'John Smith'"
1238 When the name of a field appears in an expression, the expression is
1239 applied to all the fields in the record featuring that name. So, for
1240 example, the expression:
1244 matches any record in which there is a field named 'Email' whose value
1245 terminates in (the literal string) '.org'. If we are interested in the
1246 value of some specific email, we can specify its relative position in
1247 the containing record by using "subscripts". Consider, for example:
1255 Email: mr.foo@foo.com
1260 Email: mr.foo@foo.com
1263 The regexp syntax supported in selection expressions is POSIX EREs,
1264 with several GNU extensions. *Note Regular Expressions::.
1266 3.5.2.4 Parenthesized Expressions
1267 .................................
1269 Parenthesis characters ('(' and ')') can be used to group sub
1270 expressions in the usual way.
1273 File: recutils.info, Node: SEX Operators, Next: SEX Evaluation, Prev: SEX Operands, Up: Selection Expressions
1278 The supported operators are arithmetic operators (addition, subtraction,
1279 multiplication, division and modulus), logical operators, string
1280 operators and field operators.
1282 3.5.3.1 Arithmetic Operators
1283 ............................
1285 Arithmetic operators for addition ('+'), subtraction ('-'),
1286 multiplication ('*'), integer division ('/') and modulus ('%') are
1287 supported with their usual meanings.
1289 These operators require either numeric operands or string operands
1290 whose value can be interpreted as numbers (integer or real).
1292 3.5.3.2 Boolean Operators
1293 .........................
1295 The boolean operators *and* ('&&'), *or* ('||') and *not* ('!') are
1296 supported with the same semantics as their C counterparts.
1298 A compound boolean operator '=>' is also supported in order to ease
1299 the elaboration of constraints in records: 'A => B', which can be read
1300 as "A implies B", translates into '!A || (A && B)'.
1302 The boolean operators expect integer operands, and will try to
1303 convert any string operand to an integer value.
1305 3.5.3.3 Comparison Operators
1306 ............................
1308 The compare operators *less than* ('<'), *greater than* ('>'), *less
1309 than or equal* ('<='), *greater than or equal* ('>='), *equal* ('=') and
1310 *unequal* ('!=') are supported with their usual meaning.
1312 Strings can be compared with the equality operator ('=').
1314 The match operator ('~') can be used to match a string with a given
1315 regular expression (*note Regular Expressions::).
1317 3.5.3.4 Date Comparison Operators
1318 .................................
1320 The compare operators *before* ('<<'), *after* ('>>') and *same time*
1321 ('==') can be used with fields and strings containing parseable dates.
1323 *Note Date input formats::.
1325 3.5.3.5 Field Operators
1326 .......................
1328 Field counters are replaced by the number of occurrences of a field with
1329 the given name in the record. For example:
1333 The previous expression is replaced with the number of fields named
1334 'Email' in the record. It can be zero if the record does not have a
1335 field with that name.
1337 3.5.3.6 String Operators
1338 ........................
1340 The string concatenation operator ('&') can be used to concatenate any
1341 number of strings and field values.
1343 'foo' & Name & 'bar'
1345 3.5.3.7 Conditional Operator
1346 ............................
1348 The ternary conditional operator can be used to select alternatives
1349 based on the value of some expression:
1351 expr1 ? expr2 : expr3
1353 If 'expr1' evaluates to true (i.e. it is an integer or the string
1354 representation of an integer and its value is not zero) then the
1355 operator yields 'expr2'. Otherwise it yields 'expr3'.
1358 File: recutils.info, Node: SEX Evaluation, Prev: SEX Operators, Up: Selection Expressions
1360 3.5.4 Evaluation of Selection Expressions
1361 -----------------------------------------
1365 - It is possible to refer to fields by name in selection expressions.
1366 - Records can have several fields with the same name.
1368 It is clear that some backtracking mechanism is needed in the evaluation
1369 of the selection expressions. For example, consider the following
1370 expression that is deciding whether a "registration" in a webpage should
1373 ((Email ~ "foomail\.com") || (Age <= 18)) && !#Fixed
1375 The previous expression will be evaluated for every possible
1376 permutation of the fields "Email", "Age" and "Fixed" present in the
1377 record, until one of the combinations succeeds. At that point the
1378 computation is interrupted.
1380 When used to decide whether a record matches some criteria, the goal
1381 of a selection expression is to act as a boolean expression. In that
1382 case the final value of the expression depends on both the type and the
1383 value of the result launched by the top-most subexpression:
1385 - If the result is an integer, the expression is true if its value is
1387 - If the result is a real, or a string, the expression evaluates to
1390 Sometimes a selection expression is used to compute a result instead
1391 of a boolean. In that case the returned value is converted to a string.
1392 This is used when replacing the slots in templates (*note Templates::).
1395 File: recutils.info, Node: Field Expressions, Next: Sorted Output, Prev: Selection Expressions, Up: Querying Recfiles
1397 3.6 Field Expressions
1398 =====================
1400 "Field expressions" (also known as "fexes") are a way to select fields
1401 of a record. They also allow you to do certain transformations on the
1402 selected fields, such as changing their names.
1404 A FEX comprises a sequence of "elements" separated by commas:
1406 ELEM_1,ELEM_2,...,ELEM_N
1408 Each element makes a reference to one or more fields in a record
1409 identified by a given name and an optional subscript:
1413 MIN and MAX are zero-based indexes. It is possible to refer to a field
1414 occupying a given position. For example, consider the following record:
1419 Email: mr.foo@foo.org
1421 We would select all the emails of the record with:
1425 The first email with:
1429 The third email with:
1433 The second and the third email with:
1437 And so on. It is possible to select the same field (or range of
1438 fields) more than once just by repeating them in a field expression.
1439 Thus, the field expression:
1443 will print the first email, the name, and then all the email fields
1444 including the first one.
1446 It is possible to include a "rewrite rule" in an element of a field
1447 expression, which specifies an alias for the selected fields:
1449 FIELD_NAME[MIN-MAX]:ALIAS
1451 For example, the following field expression specifies an alias for the
1452 fields named 'Email' in a record:
1454 Name,Email:ElectronicMail
1456 Since the rewrite rules only affect the fields selected in a single
1457 element of the field expression, it is possible to define different
1458 aliases to several fields having the same name but occupying different
1461 Name,Email[0]:PrimaryEmail,Email[1]:SecondaryEmail
1463 When that field expression is applied to the following record:
1466 Email: primary@email.com
1467 Email: secondary@email.com
1468 Email: other@email.com
1473 PrimaryEmail: primary@email.com
1474 SecondaryEmail: secondary@email.com
1475 Email: other@email.com
1477 It is possible to use the dot notation in order to refer to field and
1478 sub-fields. This is mainly used in the context of joins, where new
1479 fields are created having compound names such as 'Foo_Bar'. A reference
1480 to such a field can be done in the fex using dot notation as follows:
1485 File: recutils.info, Node: Sorted Output, Prev: Field Expressions, Up: Querying Recfiles
1490 This special field sets sorting criteria for the records contained in a
1491 record set. Its usage is:
1493 %sort: FIELD1 FIELD2 ...
1495 Meaning that the desired order for the records will be determined by the
1496 contents of the fields named in the '%sort' value. The sorting is
1497 always done in ascending order, and there may be records that lack the
1498 involved fields, i.e. the sorting fields need not be mandatory.
1500 It is an error to have more than one '%sort' field in the same record
1501 descriptor, as only one field list can be used as sorting criteria.
1503 Consider for example that we want to keep the records in our
1504 inventory system ordered by entry date. We could achieve that by using
1505 the following record descriptor in the database:
1513 Date: 10 February 2011
1516 Title: Ruler Pack 20
1521 As you can see in the example above, the fact we use '%sort' in a
1522 database does not mean that the database will be always physically
1523 ordered. Unsorted record sets are not a data integrity problem, and
1524 thus the diagnosis tools must not declare a recfile as +invalid because
1525 of this. The utility 'recfix' provides a way +to physically order the
1526 fields in the file (*note Invoking recfix::).
1528 On the other hand any program listing, presenting or processing data
1529 extracted from the recfile must honor the '%sort' entry. For example,
1530 when using the following 'recsel' program in the database above we would
1531 get the output sorted by date:
1533 $ recsel inventory.rec
1535 Title: Ruler Pack 20
1540 Date: 10 February 2011
1542 The sorting of the selected field depends on its type:
1544 - Numeric fields (integers, ranges, reals) are numerically ordered.
1545 - Boolean fields are ordered considering that "false" values come
1547 - Dates are ordered chronologically.
1548 - Any other kind of field is ordered using a lexicographic order.
1550 It is possible to specify several fields as the sorting criteria. In
1551 that case the records are sorted using a lexicographic order. Consider
1552 for example the following unsorted database containing marks for several
1556 %type: Class enum A B C
1579 If we wanted to sort it by 'Class' and by 'Score' we would insert a
1580 '%sort' special field in the descriptor, having:
1583 %type: Class enum A B C
1607 The order of the fields in the '%sort' field is significant. If we
1608 reverse the order in the example above then we get a different sorted
1612 %type: Class enum A B C
1636 In this last case, 'Mr. One' comes after 'Mr. Two' because the class 'A'
1637 comes before the class 'B' even though the score is the same ('6.8').
1640 File: recutils.info, Node: Editing Records, Next: Editing Fields, Prev: Querying Recfiles, Up: Top
1645 The simplest way of editing a recfile is to start your favourite text
1646 editor and hack the contents of the file as desired. However, the rec
1647 format is structured enough so recfiles can be updated automatically by
1648 programs. This is useful for writing shell scripts or when there are
1649 complex data integrity rules stored in the file that we want to be sure
1652 The following sections discuss the usage of the recutils for altering
1653 recfiles in the level of record: adding new records, deleting or
1654 commenting them out, sorting them, etc.
1658 * Inserting Records:: Inserting data into recfiles.
1659 * Deleting Records:: Removing data.
1660 * Sorting Records:: Physical reordering of records.
1663 File: recutils.info, Node: Inserting Records, Next: Deleting Records, Up: Editing Records
1665 4.1 Inserting Records
1666 =====================
1668 Adding new records to a recfile is pretty trivial: open it with your
1669 text editor and just write down the fields comprising the records. This
1670 is really the best way to add contents to a recfile containing simple
1671 data. However, complex databases may introduce some difficulties:
1673 _Multi-line values._
1674 It can be tedious to manually encode the several lines.
1676 It is difficult to manually maintain the integrity of data stored
1678 _Counters and timestamps._
1679 Some record sets feature auto-generated fields, which are commonly
1680 used to implement counters and time-stamps. *Note Auto-Generated
1683 Thus, to facilitate the insertion of new data a command line utility
1684 called 'recins' is included in the recutils. The usage of 'recins' is
1685 very simple, and can be used both in the command line or called from
1686 another program. The following subsections discuss several aspects of
1691 * Adding Records With recins:: Basics of the 'recins' utility.
1692 * Replacing Records With recins:: Substituting records in a file.
1693 * Adding Anonymous Records:: Inserting or replacing records with no
1697 File: recutils.info, Node: Adding Records With recins, Next: Replacing Records With recins, Up: Inserting Records
1699 4.1.1 Adding Records With recins
1700 --------------------------------
1702 Each invocation of 'recins' adds one record to the targeted database.
1703 The fields comprising the records are specified using pairs of '-f' and
1704 '-v' command line arguments. For example, this is how we would add the
1705 first entry to a previously empty contacts database:
1707 $ recins -f Name -v "Mr Foo" -f Email -v foo@bar.baz contacts.rec
1712 If we invoke 'recins' again on the same database we will be adding a
1715 $ recins -f Name -v "Mr Bar" -f Email -v bar@gnu.org contacts.rec
1723 There is no limit on the number of '-f' '-v' pairs that can be
1724 specified to 'recins', other than any limit on command line arguments
1725 which may be imposed by the shell.
1727 The field values provided using '-v' are encoded to follow the rec
1728 format conventions, including multi-line field values. Consider the
1731 $ recins -f Name -v "Mr. Foo" -f Address -v '
1734 Germany' contacts.rec
1742 It is also possible to provide fields already encoded as rec data for
1743 their addition, using the '-r' command line argument. This argument can
1744 be intermixed with '-f' '-v'.
1746 $ recins -f Name -v "Mr. Foo" -r "Email: foo@bar.baz" contacts.rec
1751 If the string passed to '-r' is not valid rec data then 'recins' will
1752 complain with an error and the operation will be aborted.
1754 At this time, it is not possible to add new records containing
1758 File: recutils.info, Node: Replacing Records With recins, Next: Adding Anonymous Records, Prev: Adding Records With recins, Up: Inserting Records
1760 4.1.2 Replacing Records With recins
1761 -----------------------------------
1763 'recins' can also be used to replace existing records in a database with
1764 a provided record. This is done by specifying some criteria selecting
1765 the record (or records) to be replaced.
1767 Consider for example the following command applied to our contacts
1770 $ recins -e "Email = 'foo@bar.baz'" -f Name -v "Mr. Foo" \
1771 -f Email -v "new@bar.baz" contacts.rec
1773 The contact featuring an email 'foo@bar.baz' gets replaced with the
1779 The records to be replaced can also be specified by index, or a range
1780 of indexes. For example, the following command replaces the first,
1781 second and third records in a database with dummy records:
1783 $ recins -n 0,1-2 -f Dummy -v XXX foo.rec
1791 ... Other records ...
1794 File: recutils.info, Node: Adding Anonymous Records, Prev: Replacing Records With recins, Up: Inserting Records
1796 4.1.3 Adding Anonymous Records
1797 ------------------------------
1799 In a previous chapter we noted that 'recsel' interprets the absence of a
1800 '-t' argument depending on the actual contents of the file. If the
1801 recfile contains records of just one type the command assumes that the
1802 user is referring to these records.
1804 'recins' does not follow this convention, and the absence of an
1805 explicit type always means to insert (or replace) an anonymous record.
1806 Consider for example the following database:
1809 %type: Class enum A B C
1817 If we want to insert a new mark we have to specify the type explicitly
1820 $ cat marks.rec | recins -t Marks -f Name -v Xavier -f Class -v C
1822 %type: Class enum A B C
1833 If we forget to specify the type then an anonymous record is created
1836 $ cat marks.rec | recins -f Name -v Xavier -f Class -v C
1841 %type: Class enum A B C
1850 File: recutils.info, Node: Deleting Records, Next: Sorting Records, Prev: Inserting Records, Up: Editing Records
1852 4.2 Deleting Records
1853 ====================
1855 Just as 'recins' inserts records, the utility 'recdel' deletes them.
1856 Consider the following recfile 'stock.rec':
1861 Title: First Aid Kit
1864 Title: Emergency Rations
1865 Expiry: 10 August 2009
1868 Expiry: 2 March 2009
1870 Suppose we wanted to delete all items with an 'Expiry' value before a
1871 certain date, we could do this with the following command:
1873 $ recdel -t Item -e 'Expiry << "5/12/2009"' stock.rec
1874 After running this command, only one record will remain in the file
1875 (viz: the one titled 'Emergency Rations') because all the others have
1876 expiry dates prior to 12 May 2009. (1) The '-t' option can be omitted
1877 if, and only if, there is no '%rec' field in the recfile.
1879 'recdel' tries to warn you if you attempt to perform a delete
1880 operation which it deems to be too pervasive. In such cases, it will
1881 refuse to run, unless you give the '--force' flag. However, you should
1882 not rely upon 'recdel' to protect you, because it cannot always
1883 correctly guess that you might be deleting more records than intended.
1884 For this reason, it may be wise to use the '-c' flag, which causes the
1885 relevant records to be commented out, rather than deleted. (And of
1886 course backups are always wise.)
1888 The complete options available to the 'recdel' command are explained
1889 later. *Note Invoking recdel::.
1891 ---------- Footnotes ----------
1893 (1) '5/12/2009' means the 12th day of May 2009, _not_ the fifth day
1894 of December, even if your 'LC_TIME' environment variable has been set to
1898 File: recutils.info, Node: Sorting Records, Prev: Deleting Records, Up: Editing Records
1903 In the example above, note the existence of the '%sort: Title' line.
1904 This field was discussed previously (*note Sorted Output::) and, as
1905 mentioned, does not imply that the records need to be stored in the
1906 recfile in any particular order.
1908 However, if desired, you can automatically arrange the recfile in
1909 that order using 'recfix' with the '--sort' flag. After running the
1911 $ recfix --sort stock.rec
1912 the file 'stock.rec' will have its records sorted in alphabetical order
1913 of the 'Title' fields, thus:
1918 Title: Emergency Rations
1919 Expiry: 10 August 2009
1921 Title: First Aid Kit
1925 Expiry: 2 March 2009
1928 File: recutils.info, Node: Editing Fields, Next: Field Types, Prev: Editing Records, Up: Top
1933 Fields of a recfile can, of course, be edited manually using an editor
1934 and this is often the easiest way when only a few fields need to be
1935 changed or when the nature of the changes do not follow any particular
1936 pattern. If, however, a large number of similar changes to several
1937 records are required,the 'recset' command can make the job easier.
1939 The formal description of 'recset' is presented later (*note Invoking
1940 recset::). In this chapter some typical usage examples are discussed.
1941 As with 'recdel', 'recset' if used erroneously has the potential to make
1942 very pervasive changes, which could result in a large loss of data. It
1943 is prudent therefore to take a copy of a recfile before running such
1948 * Adding Fields:: Adding new fields to records.
1949 * Setting Fields:: Editing field values.
1950 * Deleting Fields:: Removing or commenting-out fields.
1951 * Renaming Fields:: Changing the name of a field.
1954 File: recutils.info, Node: Adding Fields, Next: Setting Fields, Up: Editing Fields
1959 As mentioned above, the command 'recins' adds new records to a recfile,
1960 but it cannot add fields to an existing record. This task can be
1961 achieved automatically using 'recset' with its '-a' flag.
1963 Suppose that (after a stock inspection) you wanted to add an
1964 'Inspected' field to all the items in the recfile. The following
1965 command could be used.
1966 $ recset -t Item -f Inspected -a 'Yes' stock.rec
1967 Here, because no record selection flag was provided, the command
1968 affected _all_ the records of type 'Item'. We could limit the effect of
1969 the command using the '-e', '-q', '-n' or '-m' flags. For example to
1970 add the 'Inspected' field to only the first item the following command
1972 $ recset -t Item -n 0 -f Inspected -a 'Yes' stock.rec
1973 Similarly, a selection expression could have been used with the '-e'
1974 flag in order to add the field only to records which satisfy the
1977 If you use 'recset' with the '-a' flag on a field that already
1978 exists, a new field (in addition to those already present) will be
1979 appended with the given value.
1982 File: recutils.info, Node: Setting Fields, Next: Deleting Fields, Prev: Adding Fields, Up: Editing Fields
1987 It is also possible to update the value of a field. This is done using
1988 'recset' with its '-s' flag. In the previous example, an 'Inspected'
1989 flag was added to certain records, with the value 'yes'. After
1990 reflection, one might want to record the date of inspection, rather than
1991 a simple yes/no flag. Records which have no such field will remain
1993 $ recset -t Item -f Inspected -s '30 October 2006' stock.rec
1994 Although the above command does not have any selection criteria, it
1995 will only affect those records for which a 'Inspected' field exists.
1996 This is because the '-s' flag only sets values of existing fields. It
1997 will not create any fields.
1999 If instead the '-S' flag is used, this will create the field (if it
2000 does not already exist) _and_ set its value.
2001 $ recset -t Item -f Inspected -S '30 October 2006' stock.rec
2004 File: recutils.info, Node: Deleting Fields, Next: Renaming Fields, Prev: Setting Fields, Up: Editing Fields
2009 You can delete fields using 'recset''s '-d' flag. For example, if we
2010 wanted to delete the 'Inspected' field which we introduced above, we
2011 could do so as follows:
2012 $ recset -t Item -f Inspected -d stock.rec
2013 This would delete _all_ fields named 'Inspected' from _all_ records of
2014 type 'Item'. It may be that, we only wanted to delete the 'Inspected'
2015 fields from records which satisfy a certain condition. The following
2016 would delete the fields only from items whose 'Expiry' date was before 2
2018 $ recset -t Item -e 'Expiry << "2 January 2010"' -f Inspected -d stock.rec
2021 File: recutils.info, Node: Renaming Fields, Prev: Deleting Fields, Up: Editing Fields
2026 Another use of 'recset' is to rename existing fields. This is achieved
2027 using the '-r' flag. To rename all instances of the 'Expiry' field
2028 occurring in any record of type 'Item' to 'UseBy', the following command
2030 $ recset -t Item -f Expiry -r 'UseBy' stock.rec
2031 As with most operations, this could be done selectively, using the '-e'
2032 flag and a selection expression.
2035 File: recutils.info, Node: Field Types, Next: Constraints on Record Sets, Prev: Editing Fields, Up: Top
2040 Field values are, by default, unrestricted text strings. However, it is
2041 often useful to impose some restrictions on the values of certain
2042 fields. For example, consider the following record:
2045 Name: Jose E. Marchesi
2047 MaritalStatus: single
2048 Phone: +49 666 666 66
2050 The values of the fields must clearly follow some structure in order
2051 to make sense. 'Id' is a numeric identifier for a person. 'Name' will
2052 never use several lines. 'Age' will typically be in the range '0..120',
2053 and there are only a few valid values for 'MaritalStatus': single,
2054 married, divorced, and widow(er). Phones may be restricted to some
2055 standard format as well to be valid. All these restrictions (and many
2056 others) can be enforced by using "field types".
2058 There are two kind of field types: "anonymous" and "named". Those
2059 are described in the following subsections.
2063 * Declaring Types:: Declaration of types in record descriptors.
2064 * Types and Fields:: Associating fields with types.
2065 * Scalar Field Types:: Numbers and ranges.
2066 * String Field Types:: Lines, limited strings and regular expressions.
2067 * Enumerated Field Types:: Enumerations and boolean values.
2068 * Date and Time Types:: Dates and times.
2069 * Other Field Types:: Emails, fields, UUIDs, ...
2072 File: recutils.info, Node: Declaring Types, Next: Types and Fields, Up: Field Types
2077 A type can be declared in a record descriptor by using the '%typedef'
2078 special field. The syntax is:
2080 %typedef: TYPE_NAME TYPE_DESCRIPTION
2082 Where TYPE_NAME is the name of the new type, and TYPE_DESCRIPTION a
2083 description which varies depending of the kind of type. For example,
2084 this is how a type 'Age_t' could be defined as numbers in the range
2087 %typedef: Age_t range 0 120
2089 Type names are identifiers having the following syntax:
2091 [a-zA-Z][a-zA-Z0-9_]*
2093 Even though any identifier with that syntax could be used for types, it
2094 is a good idea to consistently follow some convention to help
2095 distinguishing type names from field names. For example, the '_t'
2096 suffix could be used for types.
2098 A type can be declared to be an alias for another type. The syntax
2101 %typedef: TYPE_NAME OTHER_TYPE_NAME
2103 Where TYPE_NAME is declared to be a synonym of OTHER_TYPE_NAME. This is
2104 useful to avoid duplicated type descriptions. For example, consider the
2108 %typedef: Item_t Id_t
2109 %typedef: Transaction_t Id_t
2111 Both 'Item_t' and 'Transaction_t' are aliases for the type 'Id_t'.
2112 Which is in turn an alias for the type 'int'. So, they are both numeric
2115 The order of the '%typedef' fields is not relevant. In particular, a
2116 type definition can forward-reference another type that is defined
2117 subsequently. The previous example could have been written as:
2119 %typedef: Item_t Id_t
2120 %typedef: Transaction_t Id_t
2123 Integrity check will complain if undefined types are referenced. As
2124 well as when any aliases up referencing back (looping back directly or
2125 indirectly) in type declarations. For example, the following set of
2126 declarations contains a loop. Thus, it's invalid:
2132 The scope of a type is the record descriptor where it is defined.
2135 File: recutils.info, Node: Types and Fields, Next: Scalar Field Types, Prev: Declaring Types, Up: Field Types
2137 6.2 Types and Fields
2138 ====================
2140 Fields can be declared to have a given type by using the '%type' special
2141 field in a record descriptor. The synopsis is:
2143 %type: FIELD_LIST TYPE_NAME_OR_DESCRIPTION
2145 Where FIELD_LIST is a list of field names separated by commas.
2146 TYPE_NAME_OR_DESCRIPTION can be either a type name which has been
2147 previously declared using '%typedef', or a type description. Type names
2148 are useful when several fields are declared to be of the same type:
2154 Anonymous types can be specified by writing a type description instead
2155 of a type name. They help to avoid superfluous type declarations in the
2156 common case where a type is used by just one field. A record containing
2157 a single 'Id' field, for example, can be defined without having to use a
2158 '%typedef' in the following way:
2164 File: recutils.info, Node: Scalar Field Types, Next: String Field Types, Prev: Types and Fields, Up: Field Types
2166 6.3 Scalar Field Types
2167 ======================
2169 The rec format supports the declaration of fields of the following
2170 scalar types: integer numbers, ranges and real numbers.
2172 Signed "integers" are supported by using the 'int' declaration:
2176 Given the declaration above, fields of type 'Id_t' must contain
2177 integers, and they may be negative. Hexadecimal values can be written
2178 using the '0x' prefix, and octal values using an extra '0'. Valid
2188 Sometimes it is desirable to reduce the "range" of integers allowed in a
2189 field. This can be achieved by using a range type declaration:
2191 %typedef: Interrupt_t range 0 15
2193 Note that it is possible to omit the minimum index in ranges. In that
2194 case it is implicitly zero:
2196 %typedef: Interrupt_t range 15
2198 It is possible to use the keywords 'MIN' and 'MAX' instead of a numeral
2199 literal in one or both of the points conforming the range. They mean
2200 the minimum and the maximum integer value supported by the
2201 implementation respectively. See the following examples:
2203 %typedef: Negative range MIN -1
2204 %typedef: Positive range 0 MAX
2205 %typedef: AnyInt range MIN MAX
2206 %typedef: Impossible range MAX MIN
2208 Hexadecimal and octal numbers can be used to specify the limits in a
2209 range. This helps to define scalar types whose natural base is not ten,
2212 %typedef: Address_t range 0x0000 0xFFFF
2213 %typedef: Perms_t range 755
2215 "Real" number fields can be declared with the 'real' type specifier. A
2216 wide range of real numbers can be represented this way, only limited by
2217 the underlying floating point representation. The decimal separator is
2218 always the dot ('.') character regardless of the locale setting. For
2221 %typedef: Longitude_t real
2223 Examples of fields of type real:
2226 %typedef: Longitude_t real
2227 %type: Width Longitude_t
2228 %type: Height Longitude_t
2234 File: recutils.info, Node: String Field Types, Next: Enumerated Field Types, Prev: Scalar Field Types, Up: Field Types
2236 6.4 String Field Types
2237 ======================
2239 The 'line' field type specifier can be used to restrict the value of a
2240 field to a single line, i.e. no newline characters are allowed. For
2241 example, a type for proper names could be declared as:
2243 %typedef: Name_t line
2245 Examples of fields of type line:
2252 Sometimes it is the maximum size of the field value that shall be
2253 restricted. The 'size' field type specifier can be used to define the
2254 maximum number of characters a field value can have. For example, if we
2255 were collecting input that will get written in a paper-based forms
2256 system allowing up to 25 characters width entries, we could declare the
2259 %typedef: Address_t size 25
2261 Note that hexadecimal and octal integer constants can also be used to
2262 specify field sizes:
2264 %typedef: Address_t size 0x18
2266 Arbitrary restrictions can be defined by using regular expressions. The
2267 "regexp" field type specifier introduces an ERE (extended regular
2268 expression) that will be matched against fields having that name. The
2271 %typedef: TYPE_NAME regexp /RE/
2273 where RE is the regular expression to match.
2275 For example, consider the 'Id_t' type designed to represent the
2276 encoding of the identifier of ID cards in some country:
2278 %typedef: Id_t regexp /[0-9]{9}[a-zA-Z]/
2280 Examples of fields of type 'Id_t' are:
2283 IDCard: invalid id card
2285 Note that the slashes delimiting the RE can be replaced with any other
2286 character that is not itself used as part of the regexp. That is useful
2287 in some cases such as:
2289 %typedef: Path_t regexp |(/[^/]/?)+|
2291 The regexp flavor supported in recfiles are the POSIX EREs plus several
2292 GNU extensions. *Note Regular Expressions::.
2295 File: recutils.info, Node: Enumerated Field Types, Next: Date and Time Types, Prev: String Field Types, Up: Field Types
2297 6.5 Enumerated Field Types
2298 ==========================
2300 Fields of this type contain symbols taken from an enumeration.
2302 The type is described by writing the sequence of symbols comprising
2303 the enumeration. Enumeration symbols are strings described by the
2306 [a-zA-Z0-9][a-zA-Z0-9_-]*
2308 The symbols are separated by blank characters (including newlines). For
2311 %typedef: Status_t enum NEW STARTED DONE CLOSED
2312 %typedef: Day_t enum Monday Tuesday Wednesday Thursday Friday
2315 It is possible to insert comments when describing an enum type. The
2316 comments are delimited by parenthesis pairs. The contents of the
2317 comments can be any character but parentheses. For example:
2319 %typedef: TaskStatus_t enum
2320 + NEW (The task was just created)
2321 + IN_PROGRESS (Task started)
2322 + CLOSED (Task closed)
2324 "Boolean" fields, declared with the type specifier 'bool', can be seen
2325 as special enumerations holding the binary values true and false.
2327 %typedef: Yesno_t bool
2329 The literals allowed in boolean fields are 'yes/no', '0/1' and
2330 'true/false'. Examples are:
2337 File: recutils.info, Node: Date and Time Types, Next: Other Field Types, Prev: Enumerated Field Types, Up: Field Types
2339 6.6 Date and Time Types
2340 =======================
2342 The "date" field type specifier can be used to declare dates and times.
2345 %typedef: TYPE_NAME date
2347 There are many permitted date formats, described in detail later in this
2348 manual (*note Date input formats::). Of particular note are the
2350 - Dates and times read from recfiles are not affected by the locale
2351 or the timezone. This means that the 'LC_TIME' and the 'TZ'
2352 environment variables are ignored. If you wish, for example, to
2353 specify a time which must be interpreted as UTC, you must
2354 explicitly append the time zone correction: e.g. '2001-1-10
2356 - The field value '1/10/2001' means January 10, 2001, *not* October
2358 - Relative times and dates (such as '1 day ago') are permitted but
2359 are not particularly useful.
2362 File: recutils.info, Node: Other Field Types, Prev: Date and Time Types, Up: Field Types
2364 6.7 Other Field Types
2365 =====================
2367 The "Email" field type specifier is used to declare electronic
2368 addresses. The synopsis is:
2370 %typedef: Email_t email
2372 Sometimes it is useful to make fields to store field names. For that
2373 purpose the "Field" field type specifier is supported. The synopsis is:
2375 %typedef: Field_t field
2377 Universally Unique Identifiers (also known as UUIDs) are a way to assign
2378 a globally unique label to some object. The "uuid" field type specifier
2379 serves that purpose. The synopsis is:
2383 The format of the uuids is specified as 32 hexadecimal digits, displayed
2384 in five groups separated by hyphens. For example:
2386 550e8400-e29b-41d4-a716-446655440000
2388 There is one other possible field type, viz: a foreign key. The
2389 following example defines the type 'Maintainer_t' to be of type "record
2390 'Hacker'"; in other words, a foreign key referring to a record in the
2391 'Hacker' record set.
2392 %typedef: Maintainer_t rec Hacker
2393 This essentially means that the values to be stored in fields of type
2394 'Maintainer_t' are of whatever type is defined for the primary key of
2395 the 'Hacker' record set. Why this is useful is discussed later. *Note
2396 Queries which Join Records::.
2399 File: recutils.info, Node: Constraints on Record Sets, Next: Checking Recfiles, Prev: Field Types, Up: Top
2401 7 Constraints on Record Sets
2402 ****************************
2404 The records in a recfile are by default not restricted to any particular
2405 structure except that they must contain one or more fields and optional
2406 comments. This provides the format with huge expressive power; but in
2407 many cases, it is also desirable to impose some restrictions in order to
2408 reflect some of the properties of the data stored in the database. It
2409 is also useful in order to preserve data integrity and thus avoid data
2412 The following sections describe the usage of some predefined special
2413 fields whose purpose is to impose this kind of restriction in the
2414 structure of the records.
2418 * Mandatory Fields:: Requiring the presence of fields.
2419 * Prohibited Fields:: Forbidding the presence of fields.
2420 * Allowed Fields:: Restricting the presence of fields.
2421 * Keys and Unique Fields:: Fields characterizing records.
2422 * Size Constraints:: Constraints on the number of records in a set.
2423 * Arbitrary Constraints:: Constraints records must comply with.
2426 File: recutils.info, Node: Mandatory Fields, Next: Prohibited Fields, Up: Constraints on Record Sets
2428 7.1 Mandatory Fields
2429 ====================
2431 Sometimes, you want to make sure that _every_ record of a particular
2432 type contains certain fields. To do this, use the special field
2433 '%mandatory'. The usage is:
2435 %mandatory: FIELD1 FIELD2 ... FIELDN
2436 The field names are separated by one or more blank characters.
2438 The fields listed in a '%mandatory' entry are non-optional; i.e. at
2439 least one field with this name shall be present in any record of this
2440 kind. Records violating this restriction are invalid and a checking
2441 tool will report the situation as a data integrity failure.
2443 Consider for example an "address book" database where each record
2444 stores the information associated with a contact. The records will be
2445 heterogeneous, in the sense they won't all contain exactly the same
2446 fields: the contact of an Internet shop will probably have a 'URL'
2447 field, while the entry for our grandmother probably won't. We still
2448 want to make sure that every entry has a field with the name of the
2449 contact. In this case, we could use '%mandatory' as follows:
2457 Name: Yoyodyne Corp.
2458 Email: sales@yoyod.com
2461 A word of caution, however: In many situations, especially in day to
2462 day social interaction, it is common to find that certain information is
2463 simply unavailable. For example, although every person has a date of
2464 birth, some people will refuse to provide that information.
2466 It is probably wise therefore to avoid stipulating a field as
2467 mandatory, unless it is essential to the enterprise. Otherwise, a data
2468 entry clerk faced with this situation will have to make the choice
2469 between dropping the entry entirely or entering some fake data to keep
2473 File: recutils.info, Node: Prohibited Fields, Next: Allowed Fields, Prev: Mandatory Fields, Up: Constraints on Record Sets
2475 7.2 Prohibited Fields
2476 =====================
2478 The inverse of '%mandatory' is '%prohibit'. Prohibited fields may not
2479 occur in _any_ record of the given type. The usage is:
2481 %prohibit: FIELD1 FIELD2 ... FIELDN
2482 The field names are separated by one or more blank characters.
2484 Fields listed in a '%prohibit' entry are forbidden; i.e. no field with
2485 this name should be present in any record of this kind. Again, records
2486 violating this restriction are invalid.
2488 Several '%prohibit' fields can appear in the same record descriptor.
2489 The set of prohibited fields is the union of all the entries. For
2490 example, in the following database both 'Id' and 'id' are prohibited:
2496 One possible use case for prohibited fields arises when some field
2497 name is reserved for some future use. For example, if we were
2498 organizing a sports competition, we would want competitors to register
2499 before the event. However a competitor's 'result' should not and cannot
2500 be entered before the competition takes place. Initially then, we would
2501 change the record descriptor as follows:
2506 At the start of the event, the '%prohibit' line can be deleted, to allow
2507 results to be entered.
2510 File: recutils.info, Node: Allowed Fields, Next: Keys and Unique Fields, Prev: Prohibited Fields, Up: Constraints on Record Sets
2515 In some cases we know the set of fields that may appear in the records
2516 of a given type, even if they are not mandatory. The '%allowed' special
2517 field is used to specify this restriction. The usage is:
2519 %allowed: FIELD1 FIELD2 ... FIELDN
2520 The field names are separated by one or more blank chracters.
2522 If there are more or one '%allowed' fields in a record descriptor, all
2523 fields of all the records in the record set must be in the union of
2524 '%allowed', '%mandatory' and '%key'. Otherwise an integrity error is
2527 Several '%allowed' fields can appear in the same record descriptor. The
2528 set of allowed fields is the union of all the entries.
2531 File: recutils.info, Node: Keys and Unique Fields, Next: Size Constraints, Prev: Allowed Fields, Up: Constraints on Record Sets
2533 7.4 Keys and Unique Fields
2534 ==========================
2536 The '%unique' and '%key' special fields are used to avoid several
2537 instances of the same field in a record, and to implement keys in record
2538 sets. Their usage is:
2540 %unique: FIELD1 FIELD2 ... FIELDN
2543 The field names are separated by one or more blank characters.
2545 Normally it is permitted for a record to contain two or more fields
2546 of the same name. The '%unique' special field revokes this
2547 permissiveness. A field declared "unique" cannot appear more than once
2550 For example, an entry in an address book database could contain an
2551 'Age' field. It does not make sense for a single person to be of
2552 several ages. So, a field could be declared as "unique" in the
2553 corresponding record descriptor as follows:
2559 Several '%unique' fields can appear in the same record descriptor. The
2560 set of unique fields is the union of all the entries.
2562 '%key' makes the referenced field the primary key of the record set.
2563 The primary key behaves as if both '%unique' and '%mandatory' had been
2564 specified for that field. Additionally, there is further restriction,
2565 viz: a given value of a primary key field may appear no more than once
2566 within a record set.
2568 Consider for example a database of items in stock. Each item is
2569 identified by a numerical 'Id' field. No item may have more than one
2570 'Id', and no items may exist without an associated 'Id'. Additionally,
2571 no two items may share the same 'Id'. This common situation can be
2572 implementing by declaring 'Id' as the key in the record descriptor:
2584 It would not make sense to have several primary keys in a record set.
2585 Thus, it is not allowed to have several '%key' fields in the same record
2586 descriptor. It is also forbidden for two items to share the same 'Id'
2587 value. Both of these situations would be data integrity violations, and
2588 will be reported by a checking tool.
2590 Elsewhere, we discuss how primary keys can be used to link one record
2591 set to another using primary keys together with foreign keys. *Note
2592 Queries which Join Records::.
2595 File: recutils.info, Node: Size Constraints, Next: Arbitrary Constraints, Prev: Keys and Unique Fields, Up: Constraints on Record Sets
2597 7.5 Size Constraints
2598 ====================
2600 Sometimes it is desirable to place constraints on entire records. This
2601 can be done with the '%size' special field which is used to limit the
2602 number of records in a record set. Its usage is:
2604 %size: [RELATIONAL_OPERATOR] NUMBER
2606 If no operator is specified then NUMBER is interpreted as the exact
2607 number of records of this type. The number can be any integer literal,
2608 including hexadecimal and octal constants. For example:
2613 + Monday Tuesday Wednesday Thursday Friday
2615 %doc: There should be exactly 7 days.
2617 The optional RELATIONAL_OPERATOR shall be one of '<', '<=', '>' and
2623 %doc: We have at most 100 different articles.
2625 It is valid to specify a size of '0', meaning that no records of this
2626 type shall exist in the file.
2628 Only one '%size' field shall appear in a record descriptor.
2631 File: recutils.info, Node: Arbitrary Constraints, Prev: Size Constraints, Up: Constraints on Record Sets
2633 7.6 Arbitrary Constraints
2634 =========================
2636 Occasionally, '%mandatory', '%prohibit' and '%size' are just not
2637 flexible enough. We might, for instance, want to ensure that _if_ a
2638 field is present, then it must have a certain relationship to other
2639 fields. Or we might want to stipulate that under certain conditions
2640 only, a record contains a particular field.
2642 To this end, recutils provides a way for arbitrary field constraints
2643 to be defined. These permit restrictions on the presence and/or value
2644 of fields, based upon the value or presence of other fields within that
2645 record. This is done using the '%constraint' special field. Its usage
2650 where EXPR is a selection expression (*note Selection Expressions::).
2651 When a constraint is present in a record set it means that all the
2652 records of that type must satisfy the selection expression, i.e. the
2653 evaluation of the expression with the record returns 1. Otherwise an
2654 integrity error is raised.
2656 Consider for example a record type 'Task' featuring two fields of
2657 type date called 'Start' and 'End'. We can use a constraint in the
2658 record set to specify that the task cannot start after it finishes:
2661 %type: Start,End date
2662 %constraint: Start << End
2664 The "implies" operator '=>' is especially useful when defining
2665 constraints, since it can be used to specify conditional constraints,
2666 i.e. constraints applying only in certain records. For example, we
2667 could specify that if a task is closed then it must have an 'End' date
2668 in the following way:
2671 %type: Start,End date
2672 %constraint: Start << End
2673 %constraint: Status = 'CLOSED' => #End
2675 It is acceptable to declare several constraints in the same record
2679 File: recutils.info, Node: Checking Recfiles, Next: Remote Descriptors, Prev: Constraints on Record Sets, Up: Top
2684 Sometimes, when creating a recfile by hand, typographical errors or
2685 other mistakes will occur. If a recfile contains such mistakes, then
2686 one cannot rely upon the results of queries or other operations.
2687 Fortunately there is a tool called 'recfix' which can find these errors.
2688 It is a good idea to get into the habit of running 'recfix' on a file
2689 after editing it, and before trying other commands.
2693 * Syntactical Errors:: Fixing structure errors in recfiles.
2694 * Semantic Errors:: Fixing semantic errors in recfiles.
2697 File: recutils.info, Node: Syntactical Errors, Next: Semantic Errors, Up: Checking Recfiles
2699 8.1 Syntactical Errors
2700 ======================
2702 One easy mistake is to forget the colon separating the field name from
2710 Running 'recfix' on this file will immediately tell us that there is a
2713 $ recfix --check inventory.rec
2714 inventory.rec: 2: error: expected a record
2715 Here, 'recfix' has diagnosed a problem in the file 'inventory.rec' and
2716 the problem lies at line 2. If, as in this case, 'recfix' shows there
2717 is a problem with the recfile, you should attend to that problem before
2718 trying to use any other recutils program on that file, otherwise strange
2719 things could happen. The '--check' flag is optional but in normal
2720 execution not required because that is the default operation.
2723 File: recutils.info, Node: Semantic Errors, Prev: Syntactical Errors, Up: Checking Recfiles
2728 However 'recfix' checks more than the syntactical integrity of the
2729 recfile. It also checks certain semantics and that the data is
2730 self-consistent. To do this, it uses the special fields of the record,
2731 some of which were introduced above (*note Constraints on Record
2732 Sets::). It is a good idea to use the special fields to stipulate the
2733 "enterprise rules" of the data.
2735 Errors will be reported if any of the following special keywords are
2736 present and the data does not match the stipulated conditions
2738 The mandated fields are missing from a record.
2740 The prohibited fields are present in a record.
2742 There is more than one field in a single record of the given name.
2744 Two or more records share the same value of the field which is the
2746 '%typedef and %type'
2747 A field has a value which does not conform to the specified type.
2749 The number of records does not conform to the specified
2752 A field does not conform to the specified constraint.
2754 An unencrypted value exists for a confidential field.
2757 File: recutils.info, Node: Remote Descriptors, Next: Grouping and Aggregates, Prev: Checking Recfiles, Up: Top
2759 9 Remote Descriptors
2760 ********************
2762 The '%rec' special field is used for two main purposes: to identify a
2763 record as a record descriptor, and to provide a name for the described
2764 record set. The synopsis of the usage of the field is the following:
2766 %rec: TYPE [URL_OR_FILE]
2768 TYPE is the name of the kind of records described by the descriptor. It
2769 is mandatory to specify it, and it follows the same lexical conventions
2770 used by field names. *Note Fields::. There is a non-enforced
2771 convention to use singular nouns, because the name makes reference to
2772 the type of a single entity, even if it applies to all the records
2773 contained in the record set. For example, the following record set
2774 contains transactions, and the type specified in the record descriptor
2785 Only one '%rec' field should be in a record descriptor. If there are
2786 more it is an integrity violation. It is highly recommended (but not
2787 enforced) to place this field in the first position of the record
2790 Sometimes it is convenient to store records of the same type in
2791 different files. The duplication of record descriptors in this case
2792 would surely lead to consistency problems. A possible solution would be
2793 to keep the record descriptor in a separated file and then include it in
2794 any operation by using pipes. For example:
2796 $ cat descriptor.rec data.rec | recsel ...
2798 For those cases it is more convenient to use a "external descriptor".
2799 External descriptors can be built appending a file path to the '%rec'
2802 %rec: FSD_Entry /path/to/file.rec
2804 The previous example indicates that a record descriptor describing
2805 the 'FSD_Entry' records shall be read from the file '/path/to/file.rec'.
2806 A record descriptor for 'FSD_Entry' may not exist in the external file.
2807 Both relative and absolute paths can be specified there.
2809 URLs can be used as sources for external descriptors as well. In
2810 that case we talk about "remote descriptors". For example:
2812 %rec: Department http://www.myorg.com/Org.rec
2814 The URL shall point to a text file containing rec data. If there is a
2815 record descriptor in the remote file documenting the 'Department' type,
2818 Note that the local record descriptor can provide additional fields
2819 to "expand" the record type. For example:
2821 %rec: FSD_Entry http://www.jemarch.net/downloads/FSD.rec
2824 The record descriptor above is including the contents of the 'FSD_Entry'
2825 record descriptor from the URL, and adding them to the local record
2826 descriptor, that in this case contains just the '%mandatory' field.
2828 If you are using GNU recutils (*note Invoking the Utilities::) to
2829 process your recfiles, any URL schema supported by 'libcurl' will work.
2832 File: recutils.info, Node: Grouping and Aggregates, Next: Queries which Join Records, Prev: Remote Descriptors, Up: Top
2834 10 Grouping and Aggregates
2835 **************************
2837 Grouping and aggregate functions are two related features which are
2838 useful to extract statistics from a record set, or a subset of that
2843 * Grouping Records:: Combining records by fields.
2844 * Aggregate Functions:: Statistics and more.
2847 File: recutils.info, Node: Grouping Records, Next: Aggregate Functions, Up: Grouping and Aggregates
2849 10.1 Grouping Records
2850 =====================
2852 Consider a recfile containing a list of items in a shop inventory. For
2853 each item it is stored its type, its category, its price, the date of
2854 the last selling operation of an item of that type, and the amount of
2855 items currently available in stock. A sample of such a database could
2861 LastSell: 20-April-2012
2867 LastSell: 22-April-2012
2873 LastSell: 22-April-2012
2879 LastSell: 21-April-2012
2888 Now imagine we are interested in grouping the contents of the 'Items'
2889 record set in groups of items of the same category. We can do it using
2890 the '-G' command line argument for 'recsel'. This argument accepts a
2891 list of fields separated by commas. The argument can be read as "group
2894 In this case we want to group by 'Category', so we would do:
2896 $ recsel -G Category
2900 LastSell: 22-April-2012
2906 LastSell: 22-April-2012
2910 LastSell: 21-April-2012
2916 LastSell: 20-April-2012
2923 We can see that the output is three records, corresponding to the three
2924 different categories of items present in the database. However, we are
2925 only interested in the types of products in each category, so we can
2926 remove unwanted information using '-p':
2928 $ recsel -G Category -p Category,Type items.rec
2940 It is also possible to group by several fields. We could group by both
2941 'Category' and 'LastSell':
2943 $ recsel -G Category,LastSell -p Category,LastSell,Type items.rec
2945 LastSell: 22-April-2012
2949 LastSell: 21-April-2012
2953 LastSell: 22-April-2012
2957 LastSell: 20-April-2012
2965 File: recutils.info, Node: Aggregate Functions, Prev: Grouping Records, Up: Grouping and Aggregates
2967 10.2 Aggregate Functions
2968 ========================
2970 recutils supports "aggregate functions". These are so called because
2971 they accept a record set and a field name as inputs and generate a
2972 single result. Usually this result is numerical.
2974 The supported aggregate functions are the following:
2977 Counts the number of occurrences of a field.
2979 Calculates the average (mean) of the numerical values of a field.
2981 Calculates the sum of the numerical values of a field.
2983 Calculates the minimum numerical value of a field.
2985 Calculates the maximum numerical value of a field.
2987 The aggregate functions are to be invoked in the field expressions in
2988 'recsel'. By default they are applied to the totality of the records in
2989 a record set. For example, using the items database from the previous
2990 section, we can do calculations as in the following examples.
2992 The SQL aggregate functions can be applied to the totality of the
2993 tuples in the relation. For example, using the 'Count' aggregate
2994 function we can calculate the number of fields named 'Category' present
2995 in the record set as follows:
2997 $ recsel -p "Count(Category)" items.rec
3000 The result is a field whose name is derived from the function name and
3001 the field passed as its parameter, separated by an underline. This name
3002 scheme probably suffices for most purposes, but it is always possible to
3003 use a rewrite rule to obtain something different:
3005 $ recsel -p "Count(Category):NumCategories" items.rec
3008 You can use different letter case in writing the name of the aggregate,
3009 and this will be reflected in the field name:
3011 $ recsel -p "CoUnT(Category)" items.rec
3014 It is possible to use more than one aggregate function in the field
3015 expression. Suppose we are also interested in the average price of the
3016 items we sell. We can use the 'Avg' aggregate:
3018 $ recsel -p "Count(Category),Avg(Price)" items.rec
3022 Now let's add a field along with an aggregate function to the field
3023 expression and see what we get:
3025 $ recsel -p "Type,Avg(Price)" items.rec
3027 Avg_Price: 12.200000
3041 We get five records! The reason is that when _only_ aggregate functions
3042 are part of the field expression, they are applied to the single record
3043 that would result from concatenating all the records in the record set
3044 together. However, when a regular field appears in the field expression
3045 the aggregate functions are applied to the individual records. This is
3046 still useful in some cases, such as a database of maintainers:
3048 Name: Jose E. Marchesi
3049 Email: jemarch@gnu.org
3050 Email: jemarch@es.gnu.org
3053 Email: positron@gnu.org
3055 Lets see how many emails each maintainer has:
3057 $ recsel -p "Name,Count(Email)" maintainers.rec
3058 Name: Jose E. Marchesi
3064 Aggregate functions are most useful when we combine them with grouping.
3065 This is when we are interested in some property of a subset of the
3066 records in the database. For example, the average prices of each item
3067 category stored in the database can be obtained by executing:
3069 $ recsel -p "Category,Avg(Price)" -G Category items.rec
3079 If we were interested in the actual prices that result in each average
3082 $ recsel -p "Category,Price,Avg(Price)" -G Category items.rec
3098 File: recutils.info, Node: Queries which Join Records, Next: Auto-Generated Fields, Prev: Grouping and Aggregates, Up: Top
3100 11 Queries which Join Records
3101 *****************************
3103 Suppose you wanted to add the residential address of the people in the
3104 'acquaintances.rec' file from *note Simple Selections::.
3106 One way to do this is as follows:
3111 Email: alf@example.com
3112 Address: 42 Abbeter Way, Inprooving, WORCS
3113 Telephone: 01234 5676789
3116 Dob: 21 February 1972
3117 Email: mandy@example.com
3118 Address: 42 Abbeter Way, Inprooving, WORCS
3119 Telephone: 01234 5676789
3123 Email: bert@example.com
3124 Address: 42 Abbeter Way, Inprooving, WORCS
3125 Telephone: 01234 5676789
3127 Name: Charles Spencer
3129 Email: charlie@example.com
3130 Address: 2 Serpe Rise, Little Worning, SURREY
3131 Telephone: 09876 5432109
3135 Email: dirk@example.com
3136 Address: 2 Serpe Rise, Little Worning, SURREY
3137 Telephone: 09876 5432109
3141 Email: ernie@example.com
3142 Address: 1 Wanter Rise, Greater Inncombe, BUCKS
3144 This will work fine. However you will notice that there are two
3145 addresses where more than one person live (presumably they are members
3146 of the same family). This has a number of disadvantages:
3147 - You have to type (or copy) the same information several times.
3148 - Should a family move house, then you would have to update the
3149 addresses (and telephone number) of all the family members.
3150 - A typing error in one of the addresses would lead an automatic
3151 query to erroneously suggest that the people lived at different
3153 - It unnecessarily increases the size of the recfile.
3157 * Foreign Keys:: Referring to records from another records.
3158 * Joining Records:: Performing cross-joins.
3161 File: recutils.info, Node: Foreign Keys, Next: Joining Records, Up: Queries which Join Records
3166 A better way would be to separate the addresses and people into
3167 different record sets. The first record set might look like this:
3171 %type: Abode rec Residence
3176 Email: alf@example.com
3180 Dob: 21 February 1972
3181 Email: mandy@example.com
3187 Email: bert@example.com
3190 Name: Charles Spencer
3192 Email: charlie@example.com
3197 Email: dirk@example.com
3206 and the second (following in the same file), like this:
3212 Address: 42 Abbeter Way, Inprooving, WORCS
3213 Telephone: 01234 5676789
3216 Address: 2 Serpe Rise, Little Worning, SURREY
3217 Telephone: 09876 5432109
3220 Address: 1 Wanter Rise, Greater Inncombe, BUCKS
3223 Here you can see that there are two record sets viz: 'Person' and
3224 'Residence'. There are six people, but only three residences, because
3225 some residences accommodate more than one person. Note also that the
3226 'Residence' descriptor has the entry '%key: Id' whilst the 'Person'
3227 descriptor has '%type: Abode rec Residence'. This is because 'Abode' is
3228 the foreign key which identifies the residence where a person lives.
3230 We could have declared the 'Id' field as '%auto'. This would have
3231 had the advantage that we need not manually update it. However, we
3232 decided that the 'Abode' field values in the 'Person' records are better
3233 as alphanumeric fields, so that they can contain human readable values.
3234 In this way, it is self-evident by reading a 'Person' record where that
3235 person lives. Yet since the 'Id' field is declared using the '%key'
3236 special field name, you can be sure that you don't accidentally reuse an
3240 File: recutils.info, Node: Joining Records, Prev: Foreign Keys, Up: Queries which Join Records
3242 11.2 Joining Records
3243 ====================
3245 The above example has also added a new field to the 'Person' record set
3246 to contain that person's mobile phone number. Note that the 'Telephone'
3247 field belongs to the 'Residence' record set because that contains the
3248 telephone number of the home, whereas 'Mobile' belongs to 'Person' since
3249 mobile telephones are normally used exclusively by one individual.
3251 If we want to look up the name and address of a person in our
3252 recfile, we can use 'recsel' as before. Because we now have more than
3253 one record set in the 'acquaintances.rec' file, we have to tell 'recsel'
3254 in which record set we want to look up records. We do this with the
3255 '-t' flag as follows:
3257 $ recsel -t Person -P Name,Abode acquaintances.rec
3276 This result tells us the names of all the people in the recfile, as
3277 well as giving a concise and hopefully effective reminder telling us
3278 where they live. However these results would not be useful to someone
3279 unacquainted with the individuals. They need a list of names and full
3280 addresses. We can use 'recsel' to produce such a list:
3282 $ recsel -t Person -j Abode acquaintances.rec
3283 Name: Charles Spencer
3285 Email: charlie@example.com
3286 Abode_Address: 2 Serpe Rise, Little Worning, SURREY
3287 Abode_Telephone: 09876 5432109
3288 Abode_Id: 2SerpeRise
3292 Email: dirk@example.com
3294 Abode_Address: 2 Serpe Rise, Little Worning, SURREY
3295 Abode_Telephone: 09876 5432109
3296 Abode_Id: 2SerpeRise
3300 Abode_Address: 1 Wanter Rise, Greater Inncombe, BUCKS
3301 Abode_Id: ChezGrampa
3303 The '-t' flag we have seen before. It tells 'recsel' that we want to
3304 extract records of type 'Person'. The '-j' flag is new. It says that
3305 we want to perform a "join". Specifically we want to join the 'Person'
3306 records according to their 'Abode' field.
3308 In the above example, 'recsel' displays several field names which do
3309 not appear anywhere in the input e.g. 'Abode_Address'. This is the
3310 'Address' field in the record joined by the foreign key 'Abode'. In
3311 this example probably only the name and address are of interest. The
3312 other information such as date of birth is incidental. The foreign key
3313 'Abode_Id' is certainly not wanted in the output since it is redundant.
3314 As usual, you can use the '-P' or '-p' options to limit the fields which
3315 will be displayed. However the full joined field name, if appropriate,
3316 must be specified. So the names and addresses without the other
3317 information can be retrieved thus:
3319 $ recsel -t Person -j Abode -p Name,Abode_Address acquaintances.rec
3320 Name: Charles Spencer
3321 Abode_Address: 2 Serpe Rise, Little Worning, SURREY
3324 Abode_Address: 2 Serpe Rise, Little Worning, SURREY
3327 Abode_Address: 1 Wanter Rise, Greater Inncombe, BUCKS
3330 File: recutils.info, Node: Auto-Generated Fields, Next: Encryption, Prev: Queries which Join Records, Up: Top
3332 12 Auto-Generated Fields
3333 ************************
3335 Consider for example a list of articles in stock in a toy store:
3340 Description: 2cm metal soldier WWII
3343 Description: Flying Helicopter Indoor Maxi
3348 It would be natural to identify the items by their descriptions, but
3349 it is also error prone: was it "Flying Helicopter Indoor Maxi" or
3350 "Flying Helicopter Maxi Indoor"? Was "Helicopter" in lower case or
3353 Thus it is quite common in databases to use some kind of numeric "Id"
3354 to uniquely identify items like those ones, because numbers are easy to
3355 increment and manipulate. So we could add a new numeric 'Id' field and
3356 use it as the primary key:
3360 %mandatory: Description
3363 Description: 2cm metal soldier WWII
3367 Description: Flying Helicopter Indoor Maxi
3372 A problem with this approach is that we must be careful to not assign
3373 already used ids when we introduce more articles in the database. Other
3374 than its uniqueness, it is not important which number is associated with
3377 To ease the management of those Ids database systems use to provide a
3378 facility called "auto-counters". Auto-counters can be implemented in
3379 recfiles using the '%auto' directive in the record descriptor. Its
3382 %auto: FIELD1 FIELD2 ... FIELDN
3384 The list of field names are separated by one or more blank characters.
3385 There can be several '%auto' fields in the same record descriptor, the
3386 effective list of auto-generated fields being the union of all the
3389 When 'recins' inserts a new record in the recfile, it looks for any
3390 declared auto field. If any of these fields are not provided explicitly
3391 in the command line then 'recins' generates them along with the
3392 user-provided fields. Such auto fields are generated at the beginning
3393 of the new records, in the same order they are found in the '%auto'
3396 For example, consider a 'items.rec' database with an empty record
3402 %mandatory: Description
3404 If we insert a new record and we do not specify an 'Id' then it will be
3405 generated automatically by 'recins':
3407 $ recins -t Item -f Description -v 'recutils t-shirts' \
3414 %mandatory: Description
3417 Description: recutils t-shirts
3420 The concrete effect of the '%auto' directive depends on the type of the
3421 affected field. The following sections document how.
3425 * Counters:: Generating incremental Ids.
3426 * Unique Identifiers:: Generating universally unique Ids.
3427 * Time-Stamps:: Tracking the creation of records.
3430 File: recutils.info, Node: Counters, Next: Unique Identifiers, Up: Auto-Generated Fields
3435 If an auto field is of type 'integer' or 'range' then any newly
3436 generated field will use the "next biggest" unused number in the record
3439 Consider the toy inventory database introduced above. We could
3440 declare the 'Id' field to be generated automatically:
3445 %mandatory: Description
3449 Description: 2cm metal soldier WWII
3452 When the next new item is introduced in the database, 'recins' will note
3453 the '%auto', and create a new 'Id' field for the new record with the
3454 next-biggest unused integer, since 'Id' is declared to be of type 'int'.
3455 In this example, the new record would have an Id of '1'. The database
3456 can still provide an explicit Id for the new record. In that case the
3457 field is not generated automatically.
3459 Note that if no explicit type is defined for an auto generated field
3460 then it is assumed to be an integer.
3463 File: recutils.info, Node: Unique Identifiers, Next: Time-Stamps, Prev: Counters, Up: Auto-Generated Fields
3465 12.2 Unique Identifiers
3466 =======================
3468 Universally Unique Identifiers, often abbreviated as UUIDs, can also be
3469 auto-generated using recutils. Suppose you maintain a database with
3470 events featuring the following record descriptor:
3474 %mandatory: Title Date
3476 What would be appropriate to identify each event? We could use an
3477 integer and declare it as auto-generated. After adding two events the
3478 database would look like this:
3482 %mandatory: Title Date
3489 Title: Dave's birthday
3492 However, suppose that we want to share our events with other people,
3493 i.e. to send them event records and to incorporate their records into
3494 our own database. In this case the 'Id's would collide. A good
3495 solution is to use 'uuids' and declare them as 'auto':
3500 %mandatory: Title Date
3502 Id: f81d4fae-7dec-11d0-a765-00a0c91e6bf6
3506 Id: f81d4fae-dc18-11d0-a765-a01328400a0c
3507 Title: Dave's birthday
3511 File: recutils.info, Node: Time-Stamps, Prev: Unique Identifiers, Up: Auto-Generated Fields
3516 Auto generated dates can be used to implement automatic timestamps.
3517 Consider for example a "Transfer" record set registering bank transfers.
3518 We want to save a timestamp every time a transfer is done, so we include
3519 an '%auto' for the date:
3528 File: recutils.info, Node: Encryption, Next: Generating Reports, Prev: Auto-Generated Fields, Up: Top
3533 For ethical or security reasons it is sometimes necessary that
3534 information in a recfile should not be readable by unauthorized people.
3535 One way to prevent a recfile from being read is to use the security
3536 features of the operating system. A more secure way would be to encrypt
3537 the entire recfile using a free strong encryption program such as GnuPG
3538 (http://gnu.org/software/gnupg). The disadvantage of both these methods
3539 is that the entire recfile has to be secured when it may well be the
3540 case that only certain data need to be protected.
3542 Recutils offers a way to encrypt specified fields in a record, whilst
3543 leaving the rest in clear text.
3547 * Confidential Fields:: Declaring fields as sensitive data.
3548 * Encrypting Files:: Encrypt confidential fields.
3549 * Decrypting Data:: Reading encrypted fields.
3552 File: recutils.info, Node: Confidential Fields, Next: Encrypting Files, Up: Encryption
3554 13.1 Confidential Fields
3555 ========================
3557 To specify that a field should be encrypted, use the '%confidential'
3558 special field. This special field declares a set of fields as
3559 "confidential", meaning they contain secret data such as passwords or
3560 personal information. Its usage is:
3562 %confidential: FIELD1 FIELD2 ... FIELDN
3564 The field names are separated by one or more blank characters. There
3565 can be several '%confidential' fields in the same record descriptor, the
3566 effective list of confidential fields being the union of all the
3569 Declaring a field as confidential indicates that its contents must
3570 not be stored in plain text, but encrypted with a password-based
3571 mechanism. When the information is retrieved from the database the
3572 confidential fields are unencrypted if the correct password is provided.
3573 Likewise, when information is inserted in the database the confidential
3574 fields are encrypted with some given password.
3576 For example, consider a database of users of some service. For each
3577 user we want to store a name, a login name, an email address and a
3578 password. All this information is public with the obvious exception of
3579 the password. Thus we declare the 'Password' field as confidential in
3580 the corresponding record descriptor:
3586 %confidential: Password
3588 The rec format does not impose the usage of a specific encryption
3589 algorithm, but requires that:
3591 - The algorithm must be password-based.
3592 - The value of any encrypted field shall begin with the string
3593 'encrypted-' followed by the encrypted data.
3594 - The encrypted data must be encoded in some ASCII encoding such as
3597 The above rules assure that it is possible to determine whether a
3598 given field is encrypted. For example, the following is an excerpt from
3599 the account database described above. It contains an entry with the
3600 password encrypted and another with the password unencrypted:
3605 Password: encrypted-AAABBBCCDDDEEEFFF
3612 Unencrypted confidential fields are a data integrity error, and
3613 utilities like 'recfix' will report it. The same utility can be used to
3614 "fix" the database by massively encrypting any unencrypted field.
3616 Nothing prevents the usage of several passwords in the same database.
3617 This allows the establishment of several level of securities or security
3618 profiles. For example, we may want to store different passwords for
3619 different online services:
3622 %confidential: WebPassword ShellPassword
3624 We could then encrypt WebPassword entries using a password shared among
3625 all the webmasters, and the ShellPassword entries with a more restricted
3626 password available only to the administrator of the machine.
3628 Note that since the utilities only accept to specify one password at
3629 a time different passwords cannot be specified at decryption time. This
3630 means that in the example above the administrator would need to run
3631 'recsel' twice in order to decrypt all the encrypted data in the
3634 The GNU recutils fully support encrypted fields. See the
3635 documentation for 'recsel', 'recins' and 'recfix' for details on how to
3636 operate on files containing confidential fields.
3639 File: recutils.info, Node: Encrypting Files, Next: Decrypting Data, Prev: Confidential Fields, Up: Encryption
3641 13.2 Encrypting Files
3642 =====================
3644 'recins' allows the insertion of encrypted fields in a database. When
3645 the '-s' ('--password') command line option is specified in the command
3646 line any field declared as confidential in the record descriptor will
3647 get encrypted using the given passphrase. If the command is executed
3648 interactively and '-s' is not used then the user is asked to provide a
3649 password using the terminal. For example, the invocation:
3651 $ recins -t Account -s mypassword -f Login -v foo -f Password \
3652 -v secret accounts.rec
3654 will encrypt the value of the 'Password' field with 'mypassword' as long
3655 as the field is declared as confidential. (*note Confidential Fields::
3656 for details on confidential fields).
3658 'recins' will issue a warning if a confidential field is inserted in
3659 the database but no password was provided to encrypt it. This is to
3660 avoid having unencrypted sensitive data in the recfiles.
3663 File: recutils.info, Node: Decrypting Data, Prev: Encrypting Files, Up: Encryption
3665 13.3 Decrypting Data
3666 ====================
3668 The contents of confidential fields can be read using the '-s'
3669 ('--password') command line option to 'recsel'. When used, any selected
3670 record containing encrypted fields will try to decrypt them with the
3671 given password. If the operation succeeds then the output will include
3672 the unencrypted data. Otherwise the ASCII-encoded encrypted data will
3675 If 'recsel' is invoked interactively and no password is specified
3676 with '-s', the user will be asked for a password in case one is needed.
3677 No echo of the password will appear in the screen. The provided
3678 password will be used to decrypt all confidential fields as if it was
3679 specified with '-s'.
3681 For example, consider the following database storing information
3682 about the user accounts of some online service. Each entry stores a
3683 login, a full name, email and a password. The password is declared as
3688 %confidential: Password
3693 Password: encrypted-AAABBBCCCDDD
3698 Password: encrypted-XXXYYYZZZUUU
3700 If we use 'recsel' to get a list of records of type 'Account' without
3701 specifying a password, or if the wrong password was specified in
3702 interactive mode, then we would get the following output with the
3705 $ cat accounts.rec | recsel -t Account -p Login,Password
3707 Password: encrypted-AAABBBCCCDDD
3710 Password: encrypted-XXXYYYZZZUUU
3712 If we specify a password and both entries were encrypted using that
3713 password, we would get the unencrypted values:
3715 $ recsel -t Account -s secret -p Login,Password accounts.rec
3722 As mentioned above, a confidential field may be encrypted with
3723 different passwords in different records (*note Confidential Fields::).
3724 For example, we may have an entry in our database with data about the
3725 account of the administrator of the online service. In that case we
3726 might want to store the password associated with that account using a
3727 different password than that for users. In that case the output of the
3728 last command would have been:
3730 $ recsel -t Account -s secret -p Login,Password accounts.rec
3738 Password: encrypted-TTTVVVBBBNNN
3740 We would need to invoke 'recsel' with the password used to encrypt the
3741 admin entry in order to read it back unencrypted.
3744 File: recutils.info, Node: Generating Reports, Next: Interoperability, Prev: Encryption, Up: Top
3746 14 Generating Reports
3747 *********************
3749 Having a list of names and addresses, one might want to use this list to
3750 address envelopes (say, to send annual greeting cards). Since addresses
3751 are normally written on several lines, it would be appropriate then to
3752 split the 'Address' field values across multiple lines as described in
3753 *note Fields::. Suitable text can now be obtained thus:
3755 $ recsel -t Person -j Abode -P Name,Abode_Address acquaintances.rec
3771 A business enterprise might want to go one step further and generate
3772 letters (such as an advertisement or a recall notice) to customers.
3773 Since 'recsel' merely selects records and fields from record sets, on
3774 its own it cannot do this; so there is another command designed for this
3775 purpose, called 'recfmt'. This command uses a "template" which defines
3776 the general form of the desired output. A letter template might look as
3783 Re: Special offer for January
3785 We are delighted to be able to offer you a 95% discount on all car and
3786 truck hire contracts between 1 January and 2 February. Please call us
3787 to take advantage of this offer.
3792 Karen van Rental (CEO)
3795 It is best to place such a template into a file, so that you can edit
3796 it as you wish. Notice the instances of double braces enclosing a field
3797 name, e.g. '{{Name}}'. These are called "spots" and indicate places
3798 where the respective field's value should be placed. Let's assume this
3799 template is in a file called 'offer.templ'. We can then pipe the output
3800 from 'recsel' into 'recfmt' in order as follows:
3802 $ recsel -t Person -j Abode acquaintances.rec | recfmt -f offer.templ
3808 Dear Charles Spencer,
3810 Re: Special offer for January
3812 We are delighted to be able to offer you a 95% discount on all car and
3817 For each record that 'recsel' selects, one copy of 'offer.templ' will be
3818 generated. Each spot will be replaced with the field value
3819 corresponding to the field name in the spot.
3823 * Templates:: Formatted output.
3826 File: recutils.info, Node: Templates, Up: Generating Reports
3831 A recfmt template is a text string that may contain "template spots".
3832 Those spots are substituted in the template using the information of a
3833 given record. Any text that is not within a spot is copied literally to
3836 Spots are written surrounded by double curly braces, like:
3840 Spots contain selection expressions, that are executed every time the
3841 template is applied to a record. The spot is then replaced by the
3842 string representation of the value returned by the expression.
3844 For example, consider the following template:
3846 Task {{Id}}: {{Summary}}
3847 ------------------------
3850 Created at {{CreatedAt}}
3852 When applied to the following record:
3855 Summary: Fix recfmt.
3856 CreatedAt: 12 December 2010
3858 + The recfmt tool shall be fixed, because right
3859 + now it is leaking 200 megabytes per processed record.
3863 Task 123: Fix recfmt.
3864 ------------------------
3865 The recfmt tool shall be fixed, because right
3866 now it is leaking 200 megabytes per processed record.
3868 Created at 12 December 2010
3870 You can use any selection expression in the slots, including
3871 conditionals and string concatenation.
3874 File: recutils.info, Node: Interoperability, Next: Bash Builtins, Prev: Generating Reports, Up: Top
3879 Included in the recutils package are a number of utilities to assist in
3880 the creation of recfiles using data which already exists in other
3881 formats, and for exporting data from recfiles so that it can be used in
3886 * CSV Files:: Converting recfiles to/from csv files.
3887 * Importing MDB Files:: Importing MS Access Databases.
3890 File: recutils.info, Node: CSV Files, Next: Importing MDB Files, Up: Interoperability
3895 Many applications are able to read and write files containing so-called
3896 "comma separated values". Such files generally contain tabular data
3897 where the columns are separated by commas and the rows by line feed
3898 and/or carriage return characters. Although record sets are not tables,
3899 tables can be easily emulated using records having the same fields in
3900 the same order. For example:
3912 In several respects records are more flexible than tables:
3914 - Fields can appear in a different order in several records.
3915 - There can be several fields with the same name in a single record.
3916 - Records can differ in the number of fields.
3918 It is evident that records, such as those in recfiles, are a more
3919 general structure than comma separated values. This means that when
3920 converting from csv files to recfiles, certain decisions need to be
3921 made. The 'rec2csv' utility (*note Invoking rec2csv::) implements an
3922 algorithm to deal with this problem and generate a table that the user
3925 The algorithm works as follows:
3927 1. The utility first scans the specified record set, building a list
3928 with the names that will become the table header.
3930 2. For each field, a header is added with the form:
3934 where N is a number in the range '2..inf' and is the "index" of the
3935 field in its containing record plus one. For example, consider the
3936 following record set:
3947 The corresponding list of headers being:
3951 3. Then duplicates are removed:
3955 4. The resulting list of headers is then used to build the table in
3956 the generated csv file.
3958 In the above example the result would be
3960 "a","b","b_2","c","d"
3961 "a1","b11","b12","c1",
3964 As shown, missing fields are implemented as empty columns in the
3968 File: recutils.info, Node: Importing MDB Files, Prev: CSV Files, Up: Interoperability
3970 15.2 Importing MDB Files
3971 ========================
3973 Access files ("mdb files") are collections of several relations, also
3974 known as tables. Tables can be either "user tables" storing user data,
3975 or "system tables" storing information such as forms, queries or the
3976 relationships between the tables.
3978 It is possible to get a listing with the names of all tables stored
3979 in a mdb file by calling 'mdb2rec' in the following way:
3981 $ mdb2rec -l sales.mdb
3986 So 'sales.mdb' stores user information in the tables Customers,
3987 Products and Orders. If we want to include system tables in the listing
3988 we can use the '-s' command line option:
3990 $ mdb2rec -s -l sales.mdb
3999 The tables with names starting with 'MSys' are system tables. The
4000 data stored in those tables is either not relevant to the recutils user
4001 (used by the Access program to create forms and the like) or is used in
4002 an indirect way by 'mdb2rec' (such as the information from
4005 Let's read some data from the 'mdb' file. We can get the relation of
4006 Products in rec format:
4008 $ mdb2rec sales.mdb Products
4010 %type: ProductID int
4011 %type: ProductName size 80
4012 %type: Discontinued bool
4015 ProductName: GNU generation T-shirt
4020 A "record descriptor" is created for the record set containing the
4021 generated records, called Products. As seen in the example, 'mdb2rec'
4022 is able to generate type information for the fields. The list of
4023 customers is similar:
4025 $ mdb2rec sales.mdb Customers
4027 %type: CustomerID size 4
4028 %type: CompanyName size 80
4029 %type: ContactName size 60
4032 CompanyName: GNU Soft
4033 ContactName: Jose E. Marchesi
4037 If no table is specified in the invocation to 'mdb2rec' all the
4038 tables in the file are processed, with the exception of the system
4039 tables, which requires '-s' to be used:
4052 File: recutils.info, Node: Bash Builtins, Next: Invoking the Utilities, Prev: Interoperability, Up: Top
4057 The command-line utilities described in *note Invoking the Utilities::
4058 are designed to be used interactively in the shell. Together, and often
4059 combined with the standard shell utilities, they provide a quite
4060 complete user interface. However, the user's experience can be greatly
4061 improved by a closer integration between the recutils and the shell.
4062 The following sections describe several extensions for 'bash', the GNU
4063 shell (*note (bash)Top::). These extensions make the shell "aware" of
4066 As with any bash built-in, help is available in the command line
4067 using the 'help' command. For example:
4071 If you installed recutils using a binary package in a GNU/Linux
4072 distribution, odds are that the built-in commands described in this
4073 chapter are already available to you. Otherwise (you get a "command not
4074 found" or similar error) you may have to register the built-in commands
4075 with your bash. This is very easy using the 'enable' bash command. The
4076 registering command for readrec would be:
4078 $ enable -f readrec.so readrec
4080 Note however that some systems require the full path to 'readrec.so'
4081 in order for this command to work.
4085 * readrec:: Exporting the contents of records to the shell.
4088 File: recutils.info, Node: readrec, Up: Bash Builtins
4093 The bash built-in 'read', when invoked with no options, consumes one
4094 line from standard input and makes it available in the predefined
4095 'REPLY' environment variable, or any other variable whose name is passed
4096 as an argument. This allows processing data structured in lines in a
4097 quite natural way. For example, the following program prints the third
4098 field of each line, with fields separated by commas, until standard
4101 # Process one line at a time.
4104 echo "The third field is " `echo $REPLY | cut -d, -f 2`
4107 However, 'read' is not very useful when it comes to processing
4108 recutils records in the shell. Even though it is possible to customize
4109 the character used by 'read' to split the input into records, we would
4110 need to ignore the empty records in the likely case of more than one
4111 empty line separating records. Also, we would need to use 'recsel' to
4112 access to the record fields. Too complicated!
4114 Thus, the 'readrec' bash built-in is similar to 'read' with the
4115 difference that it reads records instead of lines. It also "exports"
4116 the contents of the record to the user as the values of several
4117 environment variables:
4119 - 'REPLY_REC' is set to the record read from standard input.
4120 - A set of variables 'FIELD' named after each field found in the
4121 record are set to the (decoded) value of the fields found in the
4122 input record. When several fields with the same name are found in
4123 the input record then a bash array is created.
4125 Consider for example the following simple database containing
4126 contacts information:
4135 Telephone: 999666000
4138 We would like to write some shell code to send an email to all the
4139 contacts, but only if the contact has not been checked before, i.e. the
4140 'Checked' field contains 'no'. The following code snippet would do the
4141 job nicely using 'readrec':
4143 recsel contacts.rec | while readrec
4145 if [ $Checked = "no" ]
4147 mail -s "You are being checked." ${Email[0]} < email.txt
4148 recset -e "Email = '$Email'" -f Checked -S yes contacts.rec
4153 Note the usage of the bash array when accessing the primary email
4154 address of each contact. Note also that we update each contact to
4155 figure as "checked", using 'recset', so she won't get pestered again the
4156 next time the script is run.
4159 File: recutils.info, Node: Invoking the Utilities, Next: Regular Expressions, Prev: Bash Builtins, Up: Top
4161 17 Invoking the Utilities
4162 *************************
4164 Certain options are available in all of these programs. Rather than
4165 writing identical descriptions for each of the programs, they are listed
4169 Print the version number, then exit successfully.
4171 Print a help message, then exit successfully.
4173 Delimit the option list. Later arguments, if any, are treated as
4174 operands even if they begin with '-'. For example, 'recsel -- -p'
4175 reads from the file named '-p'.
4179 * Invoking recinf:: Printing information about rec files.
4180 * Invoking recsel:: Selecting records.
4181 * Invoking recins:: Inserting records.
4182 * Invoking recdel:: Deleting records.
4183 * Invoking recset:: Managing fields.
4184 * Invoking recfix:: Fixing broken rec files, and diagnostics.
4185 * Invoking recfmt:: Formatting records using templates.
4186 * Invoking csv2rec:: Converting csv data into rec data.
4187 * Invoking rec2csv:: Converting rec data into csv data.
4188 * Invoking mdb2rec:: Converting mdb files into rec files.
4191 File: recutils.info, Node: Invoking recinf, Next: Invoking recsel, Up: Invoking the Utilities
4193 17.1 Invoking recinf
4194 ====================
4196 'recinf' reads the given rec files (or the data from standard input if
4197 no file is specified) and prints a summary of the record types contained
4202 recinf [OPTION]... [FILE]...
4204 The default behavior is to emit a line per record type in the input
4205 containing its name and the number of records of that type:
4207 $ recinf hackers.rec tasks.rec
4211 If the input contains anonymous records, i.e. records that are before
4212 the first record descriptor, the corresponding output line won't have a
4218 In addition to the common options described earlier the program
4219 accepts the following options.
4223 Select records of a given type only.
4226 Print all the record descriptors present in the file.
4229 Output just the names of the record types found in the input. If
4230 the input contains only anonymous records then output nothing.
4233 Print the data in the form of sexps (Lisp expressions) instead of
4234 rec format. This option can be useful for, of course, Lisp
4238 File: recutils.info, Node: Invoking recsel, Next: Invoking recins, Prev: Invoking recinf, Up: Invoking the Utilities
4240 17.2 Invoking recsel
4241 ====================
4243 'recsel' reads the given rec files (or the data in the standard input if
4244 no file is specified) and prints out records (or part of records) based
4245 upon some criteria specified by the user.
4247 'recsel' searches rec files for records satisfying certain criteria.
4250 recsel [OPTION]... \
4251 [-n INDEXES | -e RECORD_EXPR | -q STR | -m NUM] \
4252 [-c | (-p|-P|-R) FIELD_EXPR] \
4255 If no FILE is specified then the command acts like a filter, getting
4256 the data from standard input and writing the result to standard output.
4258 In addition to the common options described earlier (*note Common
4259 Options::) the program accepts the following options.
4261 The following "global options" are available.
4264 '--case-insensitive'
4265 Make string matching case-insensitive in selection expressions.
4268 Do not section the result in records with newlines.
4270 '--include-descriptors'
4271 Print record descriptors along with the matched records.
4274 Try to decrypt confidential fields with the given password.
4277 Sort the output by the comma-separated list of field names, FIELDS.
4278 This option takes precedence over any sorting criteria specified in
4279 the corresponding record descriptor with '%sort'.
4282 Remove duplicated fields in the output records. Fields are
4283 duplicated if they have the same field name and the same value.
4286 Group the output records by the provided comma-separated list of
4287 FIELDS. Grouping is performed before sorting.
4289 The "selection options" are used to select a subset of the records in
4294 Match the records occupying the given positions in its record set.
4295 INDEXES must be a comma-separated list of numbers or ranges, with
4296 ranges being two numbers separated with dashes. For example, the
4297 following list denotes the first, the third, the fourth and all
4298 records up to the tenth: '-n 0,2,4-9'.
4301 A record selection expression (*note Selection Expressions::).
4302 Only the records matched by the expression will be taken into
4303 account to compute the output.
4306 Select records having a field whose value contains the substring
4310 Select NUM random records. If NUM is zero then select all the
4314 Select records of a given type only.
4317 Perform an inner join of the record set selected by '-t' and the
4318 record set for which FIELD is a foreign key. FIELD must be a field
4319 declared with type 'rec' and thus must be a foreign key. If a join
4320 is performed then any selection expression and field expression
4321 operate on the joined record sets.
4323 The "output options" are used to determine what information about the
4324 selected records to display to the user, and how to display it.
4328 List of fields to print for each record. NAME_LIST is a list of
4329 field names separated by commas. For example:
4332 means to print the Name and the Email of every matching record,
4333 both the field names and values.
4335 If this option is not specified then all the fields of the matching
4336 records are printed to standard output.
4338 '--print-values=NAME_LIST'
4339 Same as '-p', but print only the values of the selected fields.
4341 '--print-row=NAME_LIST'
4342 Same as '-P', but print the values separated by single spaces
4343 instead of newlines.
4346 If this option is specified then 'recsel' will print the number of
4347 matching records instead of the records themselves. This option is
4348 incompatible with '-p', '-P' and '-R'.
4350 This "special option" is available to ease the communication between
4351 the recutils and other programs, namely Lisp interpreters. This option
4352 is not intended to be used by human operators.
4355 Print the data using sexps instead of rec format.
4358 File: recutils.info, Node: Invoking recins, Next: Invoking recdel, Prev: Invoking recsel, Up: Invoking the Utilities
4360 17.3 Invoking recins
4361 ====================
4363 'recins' adds new records to a rec file or to rec data read from
4364 standard input. Synopsis:
4366 recins [OPTION]... [-t TYPE] \
4367 [-n INDEXES | -e RECORD_EXPR | -q STR | -m NUM] \
4368 [( -f STR -v STR]|[-r RECDATA )]... \
4371 The new record to be inserted by the command is constructed by using
4372 pairs of '-f' and '-v' options, or '-r'. Each pair defines a field.
4373 The order of the parameters is significant.
4375 If no FILE is specified then the command acts like a filter, getting
4376 the data from standard input and writing the result to standard output.
4378 If the specified FILE does not exist, it is created.
4380 In addition to the common options described earlier (*note Common
4381 Options::) the program accepts the following options.
4385 The type of the new record. If there is a record set in the input
4386 data matching this type then the new record is added there.
4387 Otherwise a new record set is created. If this parameter is not
4388 specified then the new record is anonymous.
4391 Declares the name of a field. This option must be followed by a
4395 The value of the field being defined.
4398 Add the fields of the record in VALUE. This option can be
4399 intermixed with '-f ... -v' pairs.
4402 Encrypt confidential fields with the given password.
4404 Don't use external record descriptors.
4406 Be verbose when reporting integrity problems.
4408 Don't generate "auto" fields. *Note Auto-Generated Fields::.
4410 Record selection arguments are supported too. If they are used then
4411 'recins' uses "replacement mode": instead of appending the new record,
4412 matched records are replaced by copies of the provided record. The
4413 selection arguments are the same as in 'recsel':
4417 Match the records occupying the given positions in its record set.
4418 INDEXES must be a comma-separated list of numbers or ranges, the
4419 ranges being two numbers separated with dashes. For example, the
4420 following list denotes the first, the third, the fourth and all
4421 records up to the tenth: '-n 0,2,4-9'.
4424 A record selection expression (*note Selection Expressions::).
4425 Matching records will get replaced.
4428 Remove records having a field whose value contains the substring
4432 Select NUM random records. If NUM is zero then all records are
4433 selected, i.e. no replace mode is activated.
4435 '--case-insensitive'
4436 Make strings case-insensitive in selection expressions.
4438 Insert the requested record even in potentially dangerous
4439 situations, such as when the data integrity of the database is
4443 File: recutils.info, Node: Invoking recdel, Next: Invoking recset, Prev: Invoking recins, Up: Invoking the Utilities
4445 17.4 Invoking recdel
4446 ====================
4448 'recdel' removes records from a rec file, or from rec data read from
4449 standard input. Synopsis:
4451 recdel [OPTIONS]... [-t TYPE] \
4452 [-n INDEXES | -e RECORD_EXPR | -q STR | -m NUM] \
4455 If no FILE is specified then the command acts like a filter, getting
4456 the data from standard input and writing the result to standard output.
4458 In addition to the common options described earlier (*note Common
4459 Options::) the program accepts the following options.
4463 Remove records of the given type. If this parameter is not
4464 specified then records of any type will be removed.
4467 Match the records occupying the given positions in its record set.
4468 INDEXES must be a comma-separated list of numbers or ranges, the
4469 ranges being two numbers separated with dashes. For example, the
4470 following list denotes the first, the third, the fourth and all
4471 records up to the tenth: '-n 0,2,4-9'.
4474 A record selection expression (*note Selection Expressions::).
4475 Only the records matched by the expression will be removed from the
4479 Remove records having a field whose value contains the substring
4483 Remove NUM random records. If NUM is zero then remove all the
4487 Comment the matching records out instead of removing them.
4489 Delete even in potentially dangerous situations, such as a request
4490 to delete all the records of some type.
4492 Don't use external record descriptors.
4494 '--case-insensitive'
4495 Make strings case-insensitive in selection expressions.
4497 Be verbose when reporting integrity problems.
4500 File: recutils.info, Node: Invoking recset, Next: Invoking recfix, Prev: Invoking recdel, Up: Invoking the Utilities
4502 17.5 Invoking recset
4503 ====================
4505 'recset' manipulates the fields of records in a rec file, or rec data
4506 read from standard input. Synopsis:
4508 recset [OPTION]... [FILE]...
4510 If no FILE is specified then the command acts like a filter, getting
4511 the data from standard input and writing the result to standard output.
4513 In addition to the common options described earlier (*note Common
4514 Options::) the program accepts the following options.
4516 Record selection options:
4519 '--case-insensitive'
4520 Make strings case-insensitive in selection expressions.
4523 Operate on the records of the given type. If this parameter is not
4524 specified then records of any type will be affected.
4527 Operate on the records occupying the given positions in its record
4528 set. INDEXES must be a comma-separated list of numbers or ranges,
4529 the ranges being two numbers separated with dashes. For example,
4530 the following list denotes the first, the third, the fourth and all
4531 records up to the tenth: '-n 0,2,4-9'.
4534 A record selection expression (*note Selection Expressions::).
4535 Only the records matched by the expression will be processed.
4538 Operate on records having a field whose value contains the
4542 Operate on NUM random records. If NUM is zero then operate on all
4545 Field selection options:
4549 Field selection expression (*note Field Expressions::) to select
4550 the fields to operate.
4556 Set the value of the selected fields to VALUE.
4559 Add a new field to the selected record with value VALUE.
4562 Set the value of the selected fields to VALUE. If some of the
4563 fields don't exist in a record, append it with the specified value.
4566 Rename a field; VALUE must be a valid field name. The field
4567 expression associated with this action must contain a single field
4568 name and an optional subscript. If an entire record set is
4569 selected then the field is renamed in the record descriptor as
4573 Delete the selected fields in the selected records.
4576 Comment out the selected fields in the selected records.
4578 Don't use external record descriptors.
4580 Be verbose when reporting integrity problems.
4582 Perform the requested operation even in potentially dangerous
4583 situations, or when the integrity of the data stored in the file is
4587 File: recutils.info, Node: Invoking recfix, Next: Invoking recfmt, Prev: Invoking recset, Up: Invoking the Utilities
4589 17.6 Invoking recfix
4590 ====================
4592 'recfix' checks and fixes rec files. Synopsis:
4594 recfix [OPTION]... [OPERATION] [OP_OPTION]... [FILE]
4596 If no FILE is specified then the command acts like a filter, getting
4597 the data from standard input and writing the result to standard output.
4599 In addition to the common options described earlier (*note Common
4600 Options::) the program accepts the following global options.
4603 Don't use external record descriptors.
4605 The effect of running 'recfix' depends on the operation it performs.
4606 The operation mode is selected by using one of the following options.
4609 Check the integrity of the database contained in the file, printing
4610 diagnostics messages in case something is not right. This is the
4613 Perform a physical sort of all the records contained in the file
4614 (or standard input) after checking for its integrity. The sorting
4615 criteria are provided by the '%sort' special field, if any. If
4616 there is an integrity failure the sorting is not performed.
4618 This is a destructive operation.
4621 Decrypt (encrypt) all the (non-)encrypted fields in the database
4622 which are marked as confidential. This operation requires a
4623 password. If no password is specified with '-s' and the program is
4624 run in a terminal, a prompt is given to get the password from the
4627 If encryption is performed on a file having encrypted fields, the
4628 operation will fail unless '--force' is used.
4630 These are destructive operations.
4632 Insert auto-generated fields as appropriate in the records which
4635 This is a destructive operation.
4637 As described above, some operations make use of these additional
4642 Password used to encrypt or decrypt fields.
4644 Force potentially dangerous operations.
4647 File: recutils.info, Node: Invoking recfmt, Next: Invoking csv2rec, Prev: Invoking recfix, Up: Invoking the Utilities
4649 17.7 Invoking recfmt
4650 ====================
4652 'recfmt' formats records using templates. Synopsis:
4654 recfmt [OPTION]... [TEMPLATE]
4656 This program always works as a filter, getting the data from the
4657 standard input and writing the result to standard output.
4659 In addition to the common options described earlier (*note Common
4660 Options::) the program accepts the following options.
4664 Read the template from the file in PATH instead of the command
4668 File: recutils.info, Node: Invoking csv2rec, Next: Invoking rec2csv, Prev: Invoking recfmt, Up: Invoking the Utilities
4670 17.8 Invoking csv2rec
4671 =====================
4673 'csv2rec' reads the given comma-separated-values file (or the data from
4674 standard input if no file is specified) and prints out the converted rec
4675 data, if possible. Synopsis:
4677 csv2rec [OPTION]... [CSV_FILE]
4679 In addition to the common options described earlier (*note Common
4680 Options::) the program accepts the following options.
4684 Type of the converted records. If no type is specified then no
4688 Be strict parsing the csv file.
4694 File: recutils.info, Node: Invoking rec2csv, Next: Invoking mdb2rec, Prev: Invoking csv2rec, Up: Invoking the Utilities
4696 17.9 Invoking rec2csv
4697 =====================
4699 'rec2csv' reads the given rec files (or the data in the standard input
4700 if no file is specified) and prints out the converted
4701 comma-separated-values. Synopsis:
4703 rec2csv [OPTION]... [REC_FILE]...
4705 The rec data can be read from files specified in the command line, or
4706 from standard input. The program writes the converted data to standard
4709 In addition to the common options described earlier (*note Common
4710 Options::) the program accepts the following options.
4714 Type of the records to convert. If no type is specified then the
4715 default records (with no name) are converted.
4718 Sort the output by the comma-separated list of field names FIELDS.
4719 This option has precedence to whatever sorting criteria are
4720 specified in the corresponding record descriptor with '%sort'.
4723 Use CHAR as the delimiter character separating fields in the
4724 output. Defaults to ','.
4727 File: recutils.info, Node: Invoking mdb2rec, Prev: Invoking rec2csv, Up: Invoking the Utilities
4729 17.10 Invoking mdb2rec
4730 ======================
4732 'mdb2rec' reads the given mdb file and prints out the converted rec
4733 data, if possible. Synopsis:
4735 mdb2rec [OPTION]... MDB_FILE [TABLE]
4737 All the tables contained in the mdb file are exported unless a table
4738 is specified in the command line.
4740 In addition to the common options described earlier (*note Common
4741 Options::) the program accepts the following options.
4745 Include system tables in the output.
4748 Dump a list of the table names contained in the mdb file, one per
4751 '--keep-empty-fields'
4752 Don't prune empty fields in the rec output.
4755 File: recutils.info, Node: Regular Expressions, Next: Date input formats, Prev: Invoking the Utilities, Up: Top
4757 18 Regular Expressions
4758 **********************
4760 The character '.' matches any single character except the null
4764 match one or more occurrences of the previous atom or regexp.
4766 match zero or one occurrences of the previous atom or regexp.
4772 Bracket expressions are used to match ranges of characters. Bracket
4773 expressions where the range is backward, for example '[z-a]', are
4774 invalid. Within square brackets, '\' is taken literally. Character
4775 classes are supported; for example '[[:digit:]]' matches a single
4778 GNU extensions are supported:
4780 matches a character within a word
4782 matches a character which is not within a word
4784 matches the beginning of a word
4786 matches the end of a word
4788 matches a word boundary
4790 matches characters which are not a word boundary
4792 matches the beginning of the whole input
4794 matches the end of the whole input
4796 Grouping is performed with parentheses '()'. An unmatched ')'
4797 matches just itself. A backslash followed by a digit acts as a
4798 back-reference and matches the same thing as the previous grouped
4799 expression indicated by that number. For example, '\2' matches the
4800 second group expression. The order of group expressions is determined
4801 by the position of their opening parenthesis '('.
4803 The alternation operator is '|'.
4805 The characters '^' and '$' always represent the beginning and end of
4806 a string respectively, except within square brackets. Within brackets,
4807 an initial '^' inverts the character class being matched.
4809 '*', '+' and '?' are special at any point in a regular expression
4810 except the following places, where they are not allowed:
4811 1. At the beginning of a regular expression
4812 2. After an open-group, '('
4813 3. After the alternation operator, '|'
4815 Intervals are specified by '{' and '}'. Invalid intervals such as
4816 'a{1z' are not accepted.
4818 The longest possible match is returned; this applies to the regular
4819 expression as a whole and (subject to this constraint) to
4820 sub-expressions within groups.
4823 File: recutils.info, Node: Date input formats, Next: GNU Free Documentation License, Prev: Regular Expressions, Up: Top
4825 19 Date input formats
4826 *********************
4830 Our units of temporal measurement, from seconds on up to months,
4831 are so complicated, asymmetrical and disjunctive so as to make
4832 coherent mental reckoning in time all but impossible. Indeed, had
4833 some tyrannical god contrived to enslave our minds to time, to make
4834 it all but impossible for us to escape subjection to sodden
4835 routines and unpleasant surprises, he could hardly have done better
4836 than handing down our present system. It is like a set of
4837 trapezoidal building blocks, with no vertical or horizontal
4838 surfaces, like a language in which the simplest thought demands
4839 ornate constructions, useless particles and lengthy
4840 circumlocutions. Unlike the more successful patterns of language
4841 and science, which enable us to face experience boldly or at least
4842 level-headedly, our system of temporal calculation silently and
4843 persistently encourages our terror of time.
4845 ... It is as though architects had to measure length in feet, width
4846 in meters and height in ells; as though basic instruction manuals
4847 demanded a knowledge of five different languages. It is no wonder
4848 then that we often look into our own immediate past or future, last
4849 Tuesday or a week from Sunday, with feelings of helpless confusion.
4852 --Robert Grudin, 'Time and the Art of Living'.
4854 This section describes the textual date representations that GNU
4855 programs accept. These are the strings you, as a user, can supply as
4856 arguments to the various programs. The C interface (via the
4857 'parse_datetime' function) is not described here.
4861 * General date syntax:: Common rules.
4862 * Calendar date items:: 19 Dec 1994.
4863 * Time of day items:: 9:20pm.
4864 * Time zone items:: EST, PDT, UTC, ...
4865 * Combined date and time of day items:: 1972-09-24T20:02:00,000000-0500.
4866 * Day of week items:: Monday and others.
4867 * Relative items in date strings:: next tuesday, 2 years ago.
4868 * Pure numbers in date strings:: 19931219, 1440.
4869 * Seconds since the Epoch:: @1078100502.
4870 * Specifying time zone rules:: TZ="America/New_York", TZ="UTC0".
4871 * Authors of parse_datetime:: Bellovin, Eggert, Salz, Berets, et al.
4874 File: recutils.info, Node: General date syntax, Next: Calendar date items, Up: Date input formats
4876 19.1 General date syntax
4877 ========================
4879 A "date" is a string, possibly empty, containing many items separated by
4880 whitespace. The whitespace may be omitted when no ambiguity arises.
4881 The empty string means the beginning of today (i.e., midnight). Order
4882 of the items is immaterial. A date string may contain many flavors of
4885 * calendar date items
4888 * combined date and time of day items
4889 * day of the week items
4893 We describe each of these item types in turn, below.
4895 A few ordinal numbers may be written out in words in some contexts.
4896 This is most useful for specifying day of the week items or relative
4897 items (see below). Among the most commonly used ordinal numbers, the
4898 word 'last' stands for -1, 'this' stands for 0, and 'first' and 'next'
4899 both stand for 1. Because the word 'second' stands for the unit of time
4900 there is no way to write the ordinal number 2, but for convenience
4901 'third' stands for 3, 'fourth' for 4, 'fifth' for 5, 'sixth' for 6,
4902 'seventh' for 7, 'eighth' for 8, 'ninth' for 9, 'tenth' for 10,
4903 'eleventh' for 11 and 'twelfth' for 12.
4905 When a month is written this way, it is still considered to be
4906 written numerically, instead of being "spelled in full"; this changes
4907 the allowed strings.
4909 In the current implementation, only English is supported for words
4910 and abbreviations like 'AM', 'DST', 'EST', 'first', 'January', 'Sunday',
4911 'tomorrow', and 'year'.
4913 The output of the 'date' command is not always acceptable as a date
4914 string, not only because of the language problem, but also because there
4915 is no standard meaning for time zone items like 'IST'. When using
4916 'date' to generate a date string intended to be parsed later, specify a
4917 date format that is independent of language and that does not use time
4918 zone items other than 'UTC' and 'Z'. Here are some ways to do this:
4920 $ LC_ALL=C TZ=UTC0 date
4921 Mon Mar 1 00:21:42 UTC 2004
4922 $ TZ=UTC0 date +'%Y-%m-%d %H:%M:%SZ'
4923 2004-03-01 00:21:42Z
4924 $ date --rfc-3339=ns # --rfc-3339 is a GNU extension.
4925 2004-02-29 16:21:42.692722128-08:00
4926 $ date --rfc-2822 # a GNU extension
4927 Sun, 29 Feb 2004 16:21:42 -0800
4928 $ date +'%Y-%m-%d %H:%M:%S %z' # %z is a GNU extension.
4929 2004-02-29 16:21:42 -0800
4930 $ date +'@%s.%N' # %s and %N are GNU extensions.
4931 @1078100502.692722128
4933 Alphabetic case is completely ignored in dates. Comments may be
4934 introduced between round parentheses, as long as included parentheses
4935 are properly nested. Hyphens not followed by a digit are currently
4936 ignored. Leading zeros on numbers are ignored.
4938 Invalid dates like '2005-02-29' or times like '24:00' are rejected.
4939 In the typical case of a host that does not support leap seconds, a time
4940 like '23:59:60' is rejected even if it corresponds to a valid leap
4944 File: recutils.info, Node: Calendar date items, Next: Time of day items, Prev: General date syntax, Up: Date input formats
4946 19.2 Calendar date items
4947 ========================
4949 A "calendar date item" specifies a day of the year. It is specified
4950 differently, depending on whether the month is specified numerically or
4951 literally. All these strings specify the same calendar date:
4953 1972-09-24 # ISO 8601.
4954 72-9-24 # Assume 19xx for 69 through 99,
4955 # 20xx for 00 through 68.
4956 72-09-24 # Leading zeros are ignored.
4957 9/24/72 # Common U.S. writing.
4959 24 Sept 72 # September has a special abbreviation.
4960 24 Sep 72 # Three-letter abbreviations always allowed.
4965 The year can also be omitted. In this case, the last specified year
4966 is used, or the current year if none. For example:
4973 For numeric months, the ISO 8601 format 'YEAR-MONTH-DAY' is allowed,
4974 where YEAR is any positive number, MONTH is a number between 01 and 12,
4975 and DAY is a number between 01 and 31. A leading zero must be present
4976 if a number is less than ten. If YEAR is 68 or smaller, then 2000 is
4977 added to it; otherwise, if YEAR is less than 100, then 1900 is added to
4978 it. The construct 'MONTH/DAY/YEAR', popular in the United States, is
4979 accepted. Also 'MONTH/DAY', omitting the year.
4981 Literal months may be spelled out in full: 'January', 'February',
4982 'March', 'April', 'May', 'June', 'July', 'August', 'September',
4983 'October', 'November' or 'December'. Literal months may be abbreviated
4984 to their first three letters, possibly followed by an abbreviating dot.
4985 It is also permitted to write 'Sept' instead of 'September'.
4987 When months are written literally, the calendar date may be given as
4988 any of the following:
4995 Or, omitting the year:
5000 File: recutils.info, Node: Time of day items, Next: Time zone items, Prev: Calendar date items, Up: Date input formats
5002 19.3 Time of day items
5003 ======================
5005 A "time of day item" in date strings specifies the time on a given day.
5006 Here are some examples, all of which represent the same time:
5011 20:02-0500 # In EST (U.S. Eastern Standard Time).
5013 More generally, the time of day may be given as 'HOUR:MINUTE:SECOND',
5014 where HOUR is a number between 0 and 23, MINUTE is a number between 0
5015 and 59, and SECOND is a number between 0 and 59 possibly followed by '.'
5016 or ',' and a fraction containing one or more digits. Alternatively,
5017 ':SECOND' can be omitted, in which case it is taken to be zero. On the
5018 rare hosts that support leap seconds, SECOND may be 60.
5020 If the time is followed by 'am' or 'pm' (or 'a.m.' or 'p.m.'), HOUR
5021 is restricted to run from 1 to 12, and ':MINUTE' may be omitted (taken
5022 to be zero). 'am' indicates the first half of the day, 'pm' indicates
5023 the second half of the day. In this notation, 12 is the predecessor of
5024 1: midnight is '12am' while noon is '12pm'. (This is the zero-oriented
5025 interpretation of '12am' and '12pm', as opposed to the old tradition
5026 derived from Latin which uses '12m' for noon and '12pm' for midnight.)
5028 The time may alternatively be followed by a time zone correction,
5029 expressed as 'SHHMM', where S is '+' or '-', HH is a number of zone
5030 hours and MM is a number of zone minutes. The zone minutes term, MM,
5031 may be omitted, in which case the one- or two-digit correction is
5032 interpreted as a number of hours. You can also separate HH from MM with
5033 a colon. When a time zone correction is given this way, it forces
5034 interpretation of the time relative to Coordinated Universal Time (UTC),
5035 overriding any previous specification for the time zone or the local
5036 time zone. For example, '+0530' and '+05:30' both stand for the time
5037 zone 5.5 hours ahead of UTC (e.g., India). This is the best way to
5038 specify a time zone correction by fractional parts of an hour. The
5039 maximum zone correction is 24 hours.
5041 Either 'am'/'pm' or a time zone correction may be specified, but not
5045 File: recutils.info, Node: Time zone items, Next: Combined date and time of day items, Prev: Time of day items, Up: Date input formats
5047 19.4 Time zone items
5048 ====================
5050 A "time zone item" specifies an international time zone, indicated by a
5051 small set of letters, e.g., 'UTC' or 'Z' for Coordinated Universal Time.
5052 Any included periods are ignored. By following a non-daylight-saving
5053 time zone by the string 'DST' in a separate word (that is, separated by
5054 some white space), the corresponding daylight saving time zone may be
5055 specified. Alternatively, a non-daylight-saving time zone can be
5056 followed by a time zone correction, to add the two values. This is
5057 normally done only for 'UTC'; for example, 'UTC+05:30' is equivalent to
5060 Time zone items other than 'UTC' and 'Z' are obsolescent and are not
5061 recommended, because they are ambiguous; for example, 'EST' has a
5062 different meaning in Australia than in the United States. Instead, it's
5063 better to use unambiguous numeric time zone corrections like '-0500', as
5064 described in the previous section.
5066 If neither a time zone item nor a time zone correction is supplied,
5067 timestamps are interpreted using the rules of the default time zone
5068 (*note Specifying time zone rules::).
5071 File: recutils.info, Node: Combined date and time of day items, Next: Day of week items, Prev: Time zone items, Up: Date input formats
5073 19.5 Combined date and time of day items
5074 ========================================
5076 The ISO 8601 date and time of day extended format consists of an ISO
5077 8601 date, a 'T' character separator, and an ISO 8601 time of day. This
5078 format is also recognized if the 'T' is replaced by a space.
5080 In this format, the time of day should use 24-hour notation.
5081 Fractional seconds are allowed, with either comma or period preceding
5082 the fraction. ISO 8601 fractional minutes and hours are not supported.
5083 Typically, hosts support nanosecond timestamp resolution; excess
5084 precision is silently discarded.
5086 Here are some examples:
5088 2012-09-24T20:02:00.052-05:00
5089 2012-12-31T23:59:59,999999999+11:00
5093 File: recutils.info, Node: Day of week items, Next: Relative items in date strings, Prev: Combined date and time of day items, Up: Date input formats
5095 19.6 Day of week items
5096 ======================
5098 The explicit mention of a day of the week will forward the date (only if
5099 necessary) to reach that day of the week in the future.
5101 Days of the week may be spelled out in full: 'Sunday', 'Monday',
5102 'Tuesday', 'Wednesday', 'Thursday', 'Friday' or 'Saturday'. Days may be
5103 abbreviated to their first three letters, optionally followed by a
5104 period. The special abbreviations 'Tues' for 'Tuesday', 'Wednes' for
5105 'Wednesday' and 'Thur' or 'Thurs' for 'Thursday' are also allowed.
5107 A number may precede a day of the week item to move forward
5108 supplementary weeks. It is best used in expression like 'third monday'.
5109 In this context, 'last DAY' or 'next DAY' is also acceptable; they move
5110 one week before or after the day that DAY by itself would represent.
5112 A comma following a day of the week item is ignored.
5115 File: recutils.info, Node: Relative items in date strings, Next: Pure numbers in date strings, Prev: Day of week items, Up: Date input formats
5117 19.7 Relative items in date strings
5118 ===================================
5120 "Relative items" adjust a date (or the current date if none) forward or
5121 backward. The effects of relative items accumulate. Here are some
5129 The unit of time displacement may be selected by the string 'year' or
5130 'month' for moving by whole years or months. These are fuzzy units, as
5131 years and months are not all of equal duration. More precise units are
5132 'fortnight' which is worth 14 days, 'week' worth 7 days, 'day' worth 24
5133 hours, 'hour' worth 60 minutes, 'minute' or 'min' worth 60 seconds, and
5134 'second' or 'sec' worth one second. An 's' suffix on these units is
5135 accepted and ignored.
5137 The unit of time may be preceded by a multiplier, given as an
5138 optionally signed number. Unsigned numbers are taken as positively
5139 signed. No number at all implies 1 for a multiplier. Following a
5140 relative item by the string 'ago' is equivalent to preceding the unit by
5141 a multiplier with value -1.
5143 The string 'tomorrow' is worth one day in the future (equivalent to
5144 'day'), the string 'yesterday' is worth one day in the past (equivalent
5147 The strings 'now' or 'today' are relative items corresponding to
5148 zero-valued time displacement, these strings come from the fact a
5149 zero-valued time displacement represents the current time when not
5150 otherwise changed by previous items. They may be used to stress other
5151 items, like in '12:00 today'. The string 'this' also has the meaning of
5152 a zero-valued time displacement, but is preferred in date strings like
5155 When a relative item causes the resulting date to cross a boundary
5156 where the clocks were adjusted, typically for daylight saving time, the
5157 resulting date and time are adjusted accordingly.
5159 The fuzz in units can cause problems with relative items. For
5160 example, '2003-07-31 -1 month' might evaluate to 2003-07-01, because
5161 2003-06-31 is an invalid date. To determine the previous month more
5162 reliably, you can ask for the month before the 15th of the current
5166 Thu, 31 Jul 2003 13:02:39 -0700
5167 $ date --date='-1 month' +'Last month was %B?'
5168 Last month was July?
5169 $ date --date="$(date +%Y-%m-15) -1 month" +'Last month was %B!'
5170 Last month was June!
5172 Also, take care when manipulating dates around clock changes such as
5173 daylight saving leaps. In a few cases these have added or subtracted as
5174 much as 24 hours from the clock, so it is often wise to adopt universal
5175 time by setting the 'TZ' environment variable to 'UTC0' before embarking
5176 on calendrical calculations.
5179 File: recutils.info, Node: Pure numbers in date strings, Next: Seconds since the Epoch, Prev: Relative items in date strings, Up: Date input formats
5181 19.8 Pure numbers in date strings
5182 =================================
5184 The precise interpretation of a pure decimal number depends on the
5185 context in the date string.
5187 If the decimal number is of the form YYYYMMDD and no other calendar
5188 date item (*note Calendar date items::) appears before it in the date
5189 string, then YYYY is read as the year, MM as the month number and DD as
5190 the day of the month, for the specified calendar date.
5192 If the decimal number is of the form HHMM and no other time of day
5193 item appears before it in the date string, then HH is read as the hour
5194 of the day and MM as the minute of the hour, for the specified time of
5195 day. MM can also be omitted.
5197 If both a calendar date and a time of day appear to the left of a
5198 number in the date string, but no relative item, then the number
5202 File: recutils.info, Node: Seconds since the Epoch, Next: Specifying time zone rules, Prev: Pure numbers in date strings, Up: Date input formats
5204 19.9 Seconds since the Epoch
5205 ============================
5207 If you precede a number with '@', it represents an internal timestamp as
5208 a count of seconds. The number can contain an internal decimal point
5209 (either '.' or ','); any excess precision not supported by the internal
5210 representation is truncated toward minus infinity. Such a number cannot
5211 be combined with any other date item, as it specifies a complete
5214 Internally, computer times are represented as a count of seconds
5215 since an epoch--a well-defined point of time. On GNU and POSIX systems,
5216 the epoch is 1970-01-01 00:00:00 UTC, so '@0' represents this time, '@1'
5217 represents 1970-01-01 00:00:01 UTC, and so forth. GNU and most other
5218 POSIX-compliant systems support such times as an extension to POSIX,
5219 using negative counts, so that '@-1' represents 1969-12-31 23:59:59 UTC.
5221 Traditional Unix systems count seconds with 32-bit two's-complement
5222 integers and can represent times from 1901-12-13 20:45:52 through
5223 2038-01-19 03:14:07 UTC. More modern systems use 64-bit counts of
5224 seconds with nanosecond subcounts, and can represent all the times in
5225 the known lifetime of the universe to a resolution of 1 nanosecond.
5227 On most hosts, these counts ignore the presence of leap seconds. For
5228 example, on most hosts '@915148799' represents 1998-12-31 23:59:59 UTC,
5229 '@915148800' represents 1999-01-01 00:00:00 UTC, and there is no way to
5230 represent the intervening leap second 1998-12-31 23:59:60 UTC.
5233 File: recutils.info, Node: Specifying time zone rules, Next: Authors of parse_datetime, Prev: Seconds since the Epoch, Up: Date input formats
5235 19.10 Specifying time zone rules
5236 ================================
5238 Normally, dates are interpreted using the rules of the current time
5239 zone, which in turn are specified by the 'TZ' environment variable, or
5240 by a system default if 'TZ' is not set. To specify a different set of
5241 default time zone rules that apply just to one date, start the date with
5242 a string of the form 'TZ="RULE"'. The two quote characters ('"') must
5243 be present in the date, and any quotes or backslashes within RULE must
5244 be escaped by a backslash.
5246 For example, with the GNU 'date' command you can answer the question
5247 "What time is it in New York when a Paris clock shows 6:30am on October
5248 31, 2004?" by using a date beginning with 'TZ="Europe/Paris"' as shown
5249 in the following shell transcript:
5251 $ export TZ="America/New_York"
5252 $ date --date='TZ="Europe/Paris" 2004-10-31 06:30'
5253 Sun Oct 31 01:30:00 EDT 2004
5255 In this example, the '--date' operand begins with its own 'TZ'
5256 setting, so the rest of that operand is processed according to
5257 'Europe/Paris' rules, treating the string '2004-10-31 06:30' as if it
5258 were in Paris. However, since the output of the 'date' command is
5259 processed according to the overall time zone rules, it uses New York
5260 time. (Paris was normally six hours ahead of New York in 2004, but this
5261 example refers to a brief Halloween period when the gap was five hours.)
5263 A 'TZ' value is a rule that typically names a location in the 'tz'
5264 database (http://www.twinsun.com/tz/tz-link.htm). A recent catalog of
5265 location names appears in the TWiki Date and Time Gateway
5266 (http://twiki.org/cgi-bin/xtra/tzdate). A few non-GNU hosts require a
5267 colon before a location name in a 'TZ' setting, e.g.,
5268 'TZ=":America/New_York"'.
5270 The 'tz' database includes a wide variety of locations ranging from
5271 'Arctic/Longyearbyen' to 'Antarctica/South_Pole', but if you are at sea
5272 and have your own private time zone, or if you are using a non-GNU host
5273 that does not support the 'tz' database, you may need to use a POSIX
5274 rule instead. Simple POSIX rules like 'UTC0' specify a time zone
5275 without daylight saving time; other rules can specify simple daylight
5276 saving regimes. *Note Specifying the Time Zone with 'TZ': (libc)TZ
5280 File: recutils.info, Node: Authors of parse_datetime, Prev: Specifying time zone rules, Up: Date input formats
5282 19.11 Authors of 'parse_datetime'
5283 =================================
5285 'parse_datetime' started life as 'getdate', as originally implemented by
5286 Steven M. Bellovin (<smb@research.att.com>) while at the University of
5287 North Carolina at Chapel Hill. The code was later tweaked by a couple
5288 of people on Usenet, then completely overhauled by Rich $alz
5289 (<rsalz@bbn.com>) and Jim Berets (<jberets@bbn.com>) in August, 1990.
5290 Various revisions for the GNU system were made by David MacKenzie, Jim
5291 Meyering, Paul Eggert and others, including renaming it to 'get_date' to
5292 avoid a conflict with the alternative Posix function 'getdate', and a
5293 later rename to 'parse_datetime'. The Posix function 'getdate' can
5294 parse more locale-specific dates using 'strptime', but relies on an
5295 environment variable and external file, and lacks the thread-safety of
5298 This chapter was originally produced by Franc,ois Pinard
5299 (<pinard@iro.umontreal.ca>) from the 'parse_datetime.y' source code, and
5300 then edited by K. Berry (<kb@cs.umb.edu>).
5303 File: recutils.info, Node: GNU Free Documentation License, Next: Concept Index, Prev: Date input formats, Up: Top
5305 Appendix A GNU Free Documentation License
5306 *****************************************
5308 Version 1.3, 3 November 2008
5310 Copyright (C) 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc.
5313 Everyone is permitted to copy and distribute verbatim copies
5314 of this license document, but changing it is not allowed.
5318 The purpose of this License is to make a manual, textbook, or other
5319 functional and useful document "free" in the sense of freedom: to
5320 assure everyone the effective freedom to copy and redistribute it,
5321 with or without modifying it, either commercially or
5322 noncommercially. Secondarily, this License preserves for the
5323 author and publisher a way to get credit for their work, while not
5324 being considered responsible for modifications made by others.
5326 This License is a kind of "copyleft", which means that derivative
5327 works of the document must themselves be free in the same sense.
5328 It complements the GNU General Public License, which is a copyleft
5329 license designed for free software.
5331 We have designed this License in order to use it for manuals for
5332 free software, because free software needs free documentation: a
5333 free program should come with manuals providing the same freedoms
5334 that the software does. But this License is not limited to
5335 software manuals; it can be used for any textual work, regardless
5336 of subject matter or whether it is published as a printed book. We
5337 recommend this License principally for works whose purpose is
5338 instruction or reference.
5340 1. APPLICABILITY AND DEFINITIONS
5342 This License applies to any manual or other work, in any medium,
5343 that contains a notice placed by the copyright holder saying it can
5344 be distributed under the terms of this License. Such a notice
5345 grants a world-wide, royalty-free license, unlimited in duration,
5346 to use that work under the conditions stated herein. The
5347 "Document", below, refers to any such manual or work. Any member
5348 of the public is a licensee, and is addressed as "you". You accept
5349 the license if you copy, modify or distribute the work in a way
5350 requiring permission under copyright law.
5352 A "Modified Version" of the Document means any work containing the
5353 Document or a portion of it, either copied verbatim, or with
5354 modifications and/or translated into another language.
5356 A "Secondary Section" is a named appendix or a front-matter section
5357 of the Document that deals exclusively with the relationship of the
5358 publishers or authors of the Document to the Document's overall
5359 subject (or to related matters) and contains nothing that could
5360 fall directly within that overall subject. (Thus, if the Document
5361 is in part a textbook of mathematics, a Secondary Section may not
5362 explain any mathematics.) The relationship could be a matter of
5363 historical connection with the subject or with related matters, or
5364 of legal, commercial, philosophical, ethical or political position
5367 The "Invariant Sections" are certain Secondary Sections whose
5368 titles are designated, as being those of Invariant Sections, in the
5369 notice that says that the Document is released under this License.
5370 If a section does not fit the above definition of Secondary then it
5371 is not allowed to be designated as Invariant. The Document may
5372 contain zero Invariant Sections. If the Document does not identify
5373 any Invariant Sections then there are none.
5375 The "Cover Texts" are certain short passages of text that are
5376 listed, as Front-Cover Texts or Back-Cover Texts, in the notice
5377 that says that the Document is released under this License. A
5378 Front-Cover Text may be at most 5 words, and a Back-Cover Text may
5379 be at most 25 words.
5381 A "Transparent" copy of the Document means a machine-readable copy,
5382 represented in a format whose specification is available to the
5383 general public, that is suitable for revising the document
5384 straightforwardly with generic text editors or (for images composed
5385 of pixels) generic paint programs or (for drawings) some widely
5386 available drawing editor, and that is suitable for input to text
5387 formatters or for automatic translation to a variety of formats
5388 suitable for input to text formatters. A copy made in an otherwise
5389 Transparent file format whose markup, or absence of markup, has
5390 been arranged to thwart or discourage subsequent modification by
5391 readers is not Transparent. An image format is not Transparent if
5392 used for any substantial amount of text. A copy that is not
5393 "Transparent" is called "Opaque".
5395 Examples of suitable formats for Transparent copies include plain
5396 ASCII without markup, Texinfo input format, LaTeX input format,
5397 SGML or XML using a publicly available DTD, and standard-conforming
5398 simple HTML, PostScript or PDF designed for human modification.
5399 Examples of transparent image formats include PNG, XCF and JPG.
5400 Opaque formats include proprietary formats that can be read and
5401 edited only by proprietary word processors, SGML or XML for which
5402 the DTD and/or processing tools are not generally available, and
5403 the machine-generated HTML, PostScript or PDF produced by some word
5404 processors for output purposes only.
5406 The "Title Page" means, for a printed book, the title page itself,
5407 plus such following pages as are needed to hold, legibly, the
5408 material this License requires to appear in the title page. For
5409 works in formats which do not have any title page as such, "Title
5410 Page" means the text near the most prominent appearance of the
5411 work's title, preceding the beginning of the body of the text.
5413 The "publisher" means any person or entity that distributes copies
5414 of the Document to the public.
5416 A section "Entitled XYZ" means a named subunit of the Document
5417 whose title either is precisely XYZ or contains XYZ in parentheses
5418 following text that translates XYZ in another language. (Here XYZ
5419 stands for a specific section name mentioned below, such as
5420 "Acknowledgements", "Dedications", "Endorsements", or "History".)
5421 To "Preserve the Title" of such a section when you modify the
5422 Document means that it remains a section "Entitled XYZ" according
5425 The Document may include Warranty Disclaimers next to the notice
5426 which states that this License applies to the Document. These
5427 Warranty Disclaimers are considered to be included by reference in
5428 this License, but only as regards disclaiming warranties: any other
5429 implication that these Warranty Disclaimers may have is void and
5430 has no effect on the meaning of this License.
5434 You may copy and distribute the Document in any medium, either
5435 commercially or noncommercially, provided that this License, the
5436 copyright notices, and the license notice saying this License
5437 applies to the Document are reproduced in all copies, and that you
5438 add no other conditions whatsoever to those of this License. You
5439 may not use technical measures to obstruct or control the reading
5440 or further copying of the copies you make or distribute. However,
5441 you may accept compensation in exchange for copies. If you
5442 distribute a large enough number of copies you must also follow the
5443 conditions in section 3.
5445 You may also lend copies, under the same conditions stated above,
5446 and you may publicly display copies.
5448 3. COPYING IN QUANTITY
5450 If you publish printed copies (or copies in media that commonly
5451 have printed covers) of the Document, numbering more than 100, and
5452 the Document's license notice requires Cover Texts, you must
5453 enclose the copies in covers that carry, clearly and legibly, all
5454 these Cover Texts: Front-Cover Texts on the front cover, and
5455 Back-Cover Texts on the back cover. Both covers must also clearly
5456 and legibly identify you as the publisher of these copies. The
5457 front cover must present the full title with all words of the title
5458 equally prominent and visible. You may add other material on the
5459 covers in addition. Copying with changes limited to the covers, as
5460 long as they preserve the title of the Document and satisfy these
5461 conditions, can be treated as verbatim copying in other respects.
5463 If the required texts for either cover are too voluminous to fit
5464 legibly, you should put the first ones listed (as many as fit
5465 reasonably) on the actual cover, and continue the rest onto
5468 If you publish or distribute Opaque copies of the Document
5469 numbering more than 100, you must either include a machine-readable
5470 Transparent copy along with each Opaque copy, or state in or with
5471 each Opaque copy a computer-network location from which the general
5472 network-using public has access to download using public-standard
5473 network protocols a complete Transparent copy of the Document, free
5474 of added material. If you use the latter option, you must take
5475 reasonably prudent steps, when you begin distribution of Opaque
5476 copies in quantity, to ensure that this Transparent copy will
5477 remain thus accessible at the stated location until at least one
5478 year after the last time you distribute an Opaque copy (directly or
5479 through your agents or retailers) of that edition to the public.
5481 It is requested, but not required, that you contact the authors of
5482 the Document well before redistributing any large number of copies,
5483 to give them a chance to provide you with an updated version of the
5488 You may copy and distribute a Modified Version of the Document
5489 under the conditions of sections 2 and 3 above, provided that you
5490 release the Modified Version under precisely this License, with the
5491 Modified Version filling the role of the Document, thus licensing
5492 distribution and modification of the Modified Version to whoever
5493 possesses a copy of it. In addition, you must do these things in
5494 the Modified Version:
5496 A. Use in the Title Page (and on the covers, if any) a title
5497 distinct from that of the Document, and from those of previous
5498 versions (which should, if there were any, be listed in the
5499 History section of the Document). You may use the same title
5500 as a previous version if the original publisher of that
5501 version gives permission.
5503 B. List on the Title Page, as authors, one or more persons or
5504 entities responsible for authorship of the modifications in
5505 the Modified Version, together with at least five of the
5506 principal authors of the Document (all of its principal
5507 authors, if it has fewer than five), unless they release you
5508 from this requirement.
5510 C. State on the Title page the name of the publisher of the
5511 Modified Version, as the publisher.
5513 D. Preserve all the copyright notices of the Document.
5515 E. Add an appropriate copyright notice for your modifications
5516 adjacent to the other copyright notices.
5518 F. Include, immediately after the copyright notices, a license
5519 notice giving the public permission to use the Modified
5520 Version under the terms of this License, in the form shown in
5523 G. Preserve in that license notice the full lists of Invariant
5524 Sections and required Cover Texts given in the Document's
5527 H. Include an unaltered copy of this License.
5529 I. Preserve the section Entitled "History", Preserve its Title,
5530 and add to it an item stating at least the title, year, new
5531 authors, and publisher of the Modified Version as given on the
5532 Title Page. If there is no section Entitled "History" in the
5533 Document, create one stating the title, year, authors, and
5534 publisher of the Document as given on its Title Page, then add
5535 an item describing the Modified Version as stated in the
5538 J. Preserve the network location, if any, given in the Document
5539 for public access to a Transparent copy of the Document, and
5540 likewise the network locations given in the Document for
5541 previous versions it was based on. These may be placed in the
5542 "History" section. You may omit a network location for a work
5543 that was published at least four years before the Document
5544 itself, or if the original publisher of the version it refers
5545 to gives permission.
5547 K. For any section Entitled "Acknowledgements" or "Dedications",
5548 Preserve the Title of the section, and preserve in the section
5549 all the substance and tone of each of the contributor
5550 acknowledgements and/or dedications given therein.
5552 L. Preserve all the Invariant Sections of the Document, unaltered
5553 in their text and in their titles. Section numbers or the
5554 equivalent are not considered part of the section titles.
5556 M. Delete any section Entitled "Endorsements". Such a section
5557 may not be included in the Modified Version.
5559 N. Do not retitle any existing section to be Entitled
5560 "Endorsements" or to conflict in title with any Invariant
5563 O. Preserve any Warranty Disclaimers.
5565 If the Modified Version includes new front-matter sections or
5566 appendices that qualify as Secondary Sections and contain no
5567 material copied from the Document, you may at your option designate
5568 some or all of these sections as invariant. To do this, add their
5569 titles to the list of Invariant Sections in the Modified Version's
5570 license notice. These titles must be distinct from any other
5573 You may add a section Entitled "Endorsements", provided it contains
5574 nothing but endorsements of your Modified Version by various
5575 parties--for example, statements of peer review or that the text
5576 has been approved by an organization as the authoritative
5577 definition of a standard.
5579 You may add a passage of up to five words as a Front-Cover Text,
5580 and a passage of up to 25 words as a Back-Cover Text, to the end of
5581 the list of Cover Texts in the Modified Version. Only one passage
5582 of Front-Cover Text and one of Back-Cover Text may be added by (or
5583 through arrangements made by) any one entity. If the Document
5584 already includes a cover text for the same cover, previously added
5585 by you or by arrangement made by the same entity you are acting on
5586 behalf of, you may not add another; but you may replace the old
5587 one, on explicit permission from the previous publisher that added
5590 The author(s) and publisher(s) of the Document do not by this
5591 License give permission to use their names for publicity for or to
5592 assert or imply endorsement of any Modified Version.
5594 5. COMBINING DOCUMENTS
5596 You may combine the Document with other documents released under
5597 this License, under the terms defined in section 4 above for
5598 modified versions, provided that you include in the combination all
5599 of the Invariant Sections of all of the original documents,
5600 unmodified, and list them all as Invariant Sections of your
5601 combined work in its license notice, and that you preserve all
5602 their Warranty Disclaimers.
5604 The combined work need only contain one copy of this License, and
5605 multiple identical Invariant Sections may be replaced with a single
5606 copy. If there are multiple Invariant Sections with the same name
5607 but different contents, make the title of each such section unique
5608 by adding at the end of it, in parentheses, the name of the
5609 original author or publisher of that section if known, or else a
5610 unique number. Make the same adjustment to the section titles in
5611 the list of Invariant Sections in the license notice of the
5614 In the combination, you must combine any sections Entitled
5615 "History" in the various original documents, forming one section
5616 Entitled "History"; likewise combine any sections Entitled
5617 "Acknowledgements", and any sections Entitled "Dedications". You
5618 must delete all sections Entitled "Endorsements."
5620 6. COLLECTIONS OF DOCUMENTS
5622 You may make a collection consisting of the Document and other
5623 documents released under this License, and replace the individual
5624 copies of this License in the various documents with a single copy
5625 that is included in the collection, provided that you follow the
5626 rules of this License for verbatim copying of each of the documents
5627 in all other respects.
5629 You may extract a single document from such a collection, and
5630 distribute it individually under this License, provided you insert
5631 a copy of this License into the extracted document, and follow this
5632 License in all other respects regarding verbatim copying of that
5635 7. AGGREGATION WITH INDEPENDENT WORKS
5637 A compilation of the Document or its derivatives with other
5638 separate and independent documents or works, in or on a volume of a
5639 storage or distribution medium, is called an "aggregate" if the
5640 copyright resulting from the compilation is not used to limit the
5641 legal rights of the compilation's users beyond what the individual
5642 works permit. When the Document is included in an aggregate, this
5643 License does not apply to the other works in the aggregate which
5644 are not themselves derivative works of the Document.
5646 If the Cover Text requirement of section 3 is applicable to these
5647 copies of the Document, then if the Document is less than one half
5648 of the entire aggregate, the Document's Cover Texts may be placed
5649 on covers that bracket the Document within the aggregate, or the
5650 electronic equivalent of covers if the Document is in electronic
5651 form. Otherwise they must appear on printed covers that bracket
5652 the whole aggregate.
5656 Translation is considered a kind of modification, so you may
5657 distribute translations of the Document under the terms of section
5658 4. Replacing Invariant Sections with translations requires special
5659 permission from their copyright holders, but you may include
5660 translations of some or all Invariant Sections in addition to the
5661 original versions of these Invariant Sections. You may include a
5662 translation of this License, and all the license notices in the
5663 Document, and any Warranty Disclaimers, provided that you also
5664 include the original English version of this License and the
5665 original versions of those notices and disclaimers. In case of a
5666 disagreement between the translation and the original version of
5667 this License or a notice or disclaimer, the original version will
5670 If a section in the Document is Entitled "Acknowledgements",
5671 "Dedications", or "History", the requirement (section 4) to
5672 Preserve its Title (section 1) will typically require changing the
5677 You may not copy, modify, sublicense, or distribute the Document
5678 except as expressly provided under this License. Any attempt
5679 otherwise to copy, modify, sublicense, or distribute it is void,
5680 and will automatically terminate your rights under this License.
5682 However, if you cease all violation of this License, then your
5683 license from a particular copyright holder is reinstated (a)
5684 provisionally, unless and until the copyright holder explicitly and
5685 finally terminates your license, and (b) permanently, if the
5686 copyright holder fails to notify you of the violation by some
5687 reasonable means prior to 60 days after the cessation.
5689 Moreover, your license from a particular copyright holder is
5690 reinstated permanently if the copyright holder notifies you of the
5691 violation by some reasonable means, this is the first time you have
5692 received notice of violation of this License (for any work) from
5693 that copyright holder, and you cure the violation prior to 30 days
5694 after your receipt of the notice.
5696 Termination of your rights under this section does not terminate
5697 the licenses of parties who have received copies or rights from you
5698 under this License. If your rights have been terminated and not
5699 permanently reinstated, receipt of a copy of some or all of the
5700 same material does not give you any rights to use it.
5702 10. FUTURE REVISIONS OF THIS LICENSE
5704 The Free Software Foundation may publish new, revised versions of
5705 the GNU Free Documentation License from time to time. Such new
5706 versions will be similar in spirit to the present version, but may
5707 differ in detail to address new problems or concerns. See
5708 <http://www.gnu.org/copyleft/>.
5710 Each version of the License is given a distinguishing version
5711 number. If the Document specifies that a particular numbered
5712 version of this License "or any later version" applies to it, you
5713 have the option of following the terms and conditions either of
5714 that specified version or of any later version that has been
5715 published (not as a draft) by the Free Software Foundation. If the
5716 Document does not specify a version number of this License, you may
5717 choose any version ever published (not as a draft) by the Free
5718 Software Foundation. If the Document specifies that a proxy can
5719 decide which future versions of this License can be used, that
5720 proxy's public statement of acceptance of a version permanently
5721 authorizes you to choose that version for the Document.
5725 "Massive Multiauthor Collaboration Site" (or "MMC Site") means any
5726 World Wide Web server that publishes copyrightable works and also
5727 provides prominent facilities for anybody to edit those works. A
5728 public wiki that anybody can edit is an example of such a server.
5729 A "Massive Multiauthor Collaboration" (or "MMC") contained in the
5730 site means any set of copyrightable works thus published on the MMC
5733 "CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0
5734 license published by Creative Commons Corporation, a not-for-profit
5735 corporation with a principal place of business in San Francisco,
5736 California, as well as future copyleft versions of that license
5737 published by that same organization.
5739 "Incorporate" means to publish or republish a Document, in whole or
5740 in part, as part of another Document.
5742 An MMC is "eligible for relicensing" if it is licensed under this
5743 License, and if all works that were first published under this
5744 License somewhere other than this MMC, and subsequently
5745 incorporated in whole or in part into the MMC, (1) had no cover
5746 texts or invariant sections, and (2) were thus incorporated prior
5747 to November 1, 2008.
5749 The operator of an MMC Site may republish an MMC contained in the
5750 site under CC-BY-SA on the same site at any time before August 1,
5751 2009, provided the MMC is eligible for relicensing.
5753 ADDENDUM: How to use this License for your documents
5754 ====================================================
5756 To use this License in a document you have written, include a copy of
5757 the License in the document and put the following copyright and license
5758 notices just after the title page:
5760 Copyright (C) YEAR YOUR NAME.
5761 Permission is granted to copy, distribute and/or modify this document
5762 under the terms of the GNU Free Documentation License, Version 1.3
5763 or any later version published by the Free Software Foundation;
5764 with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
5765 Texts. A copy of the license is included in the section entitled ``GNU
5766 Free Documentation License''.
5768 If you have Invariant Sections, Front-Cover Texts and Back-Cover
5769 Texts, replace the "with...Texts." line with this:
5771 with the Invariant Sections being LIST THEIR TITLES, with
5772 the Front-Cover Texts being LIST, and with the Back-Cover Texts
5775 If you have Invariant Sections without Cover Texts, or some other
5776 combination of the three, merge those two alternatives to suit the
5779 If your document contains nontrivial examples of program code, we
5780 recommend releasing these examples in parallel under your choice of free
5781 software license, such as the GNU General Public License, to permit
5782 their use in free software.
5785 File: recutils.info, Node: Concept Index, Prev: GNU Free Documentation License, Up: Top
5793 * %allowed: Allowed Fields. (line 6)
5794 * %auto: Auto-Generated Fields.
5796 * %confidential: Confidential Fields. (line 6)
5797 * %constraint: Arbitrary Constraints.
5799 * %doc: Documenting Records. (line 6)
5800 * %key: Keys and Unique Fields.
5802 * %key <1>: Foreign Keys. (line 65)
5803 * %key <2>: Auto-Generated Fields.
5805 * %mandatory: Record Sets Properties.
5807 * %mandatory <1>: Mandatory Fields. (line 6)
5808 * %prohibit: Prohibited Fields. (line 6)
5809 * %rec: Record Sets. (line 9)
5810 * %rec <1>: Remote Descriptors. (line 6)
5811 * %size: Size Constraints. (line 6)
5812 * %sort: Sorted Output. (line 6)
5813 * %type: Types and Fields. (line 6)
5814 * %typedef: Types and Fields. (line 6)
5815 * %unique: Keys and Unique Fields.
5817 * abbreviations for months: Calendar date items. (line 38)
5818 * adding fields: Adding Fields. (line 6)
5819 * aggregate function: Aggregate Functions. (line 6)
5820 * aliasing, field name aliasing: Field Expressions. (line 52)
5821 * allowed fields: Allowed Fields. (line 6)
5822 * anonymous types: Types and Fields. (line 20)
5823 * arithmetic operators: SEX Operators. (line 13)
5824 * authors of parse_datetime: Authors of parse_datetime.
5826 * automatically generated values: Auto-Generated Fields.
5828 * bash: Bash Builtins. (line 6)
5829 * beginning of time, for POSIX: Seconds since the Epoch.
5831 * Bellovin, Steven M.: Authors of parse_datetime.
5833 * Berets, Jim: Authors of parse_datetime.
5835 * Berry, K.: Authors of parse_datetime.
5837 * books: A Little Example. (line 6)
5838 * boolean operators: SEX Operators. (line 23)
5839 * boolean types: Enumerated Field Types.
5841 * calendar date item: Calendar date items. (line 6)
5842 * case, ignored in dates: General date syntax. (line 60)
5843 * case, in field names: Fields. (line 22)
5844 * case, in selection expressions: Invoking recsel. (line 28)
5845 * case, in selection expressions <1>: Invoking recins. (line 79)
5846 * checking recfiles: Invoking recfix. (line 6)
5847 * combined date and time of day item: Combined date and time of day items.
5849 * comma separated values: CSV Files. (line 6)
5850 * comma separated values <1>: Invoking csv2rec. (line 6)
5851 * comma separated values <2>: Invoking rec2csv. (line 6)
5852 * comments: Comments. (line 6)
5853 * comments, in dates: General date syntax. (line 60)
5854 * comments, in enumerated types: Enumerated Field Types.
5856 * comparison: SEX Operators. (line 36)
5857 * compulsory fields: Mandatory Fields. (line 6)
5858 * conditional operator: SEX Operators. (line 76)
5859 * confidential data: Confidential Fields. (line 6)
5860 * constraints: Arbitrary Constraints.
5862 * counters: Counters. (line 6)
5863 * counting occurrences of a field: SEX Operators. (line 56)
5864 * csv: CSV Files. (line 6)
5865 * csv <1>: Invoking csv2rec. (line 6)
5866 * csv <2>: Invoking rec2csv. (line 6)
5867 * csv2rec: Invoking csv2rec. (line 6)
5868 * date and time of day format, ISO 8601: Combined date and time of day items.
5870 * date comparison: Selecting by predicate.
5872 * date comparison <1>: Selecting by predicate.
5874 * date comparison <2>: SEX Operators. (line 48)
5875 * date format, ISO 8601: Calendar date items. (line 30)
5876 * date input formats: Date input formats. (line 6)
5877 * date, fields containing dates: Date and Time Types. (line 6)
5878 * day of week: Enumerated Field Types.
5880 * day of week item: Day of week items. (line 6)
5881 * decimal separator: Scalar Field Types. (line 54)
5882 * default record types: Record Sets. (line 63)
5883 * deleting fields: Deleting Fields. (line 6)
5884 * deleting records: Deleting Records. (line 6)
5885 * deleting records <1>: Invoking recdel. (line 6)
5886 * description of record sets: Documenting Records. (line 6)
5887 * descriptor: Record Descriptors. (line 6)
5888 * descriptor, external descriptor: Remote Descriptors. (line 42)
5889 * displacement of dates: Relative items in date strings.
5891 * documentation fields: Documenting Records. (line 6)
5892 * duplication, avoiding: Foreign Keys. (line 7)
5893 * editing fields: Invoking recset. (line 6)
5894 * Eggert, Paul: Authors of parse_datetime.
5896 * email: Other Field Types. (line 6)
5897 * encrypted fields: Confidential Fields. (line 18)
5898 * encryption: Encryption. (line 6)
5899 * enumerated types: Enumerated Field Types.
5901 * epoch, for POSIX: Seconds since the Epoch.
5903 * evaluation, of selection expressions: SEX Evaluation. (line 6)
5904 * external descriptor: Remote Descriptors. (line 42)
5905 * FEX: Field Expressions. (line 6)
5906 * field: Fields. (line 6)
5907 * field expressions: Field Expressions. (line 6)
5908 * field name: Fields. (line 16)
5909 * field operators: SEX Operators. (line 56)
5910 * field size: String Field Types. (line 19)
5911 * field types,: Types and Fields. (line 6)
5912 * field values: Fields. (line 34)
5913 * field values, in selection expressions: SEX Operands. (line 51)
5914 * field, allowed fields: Allowed Fields. (line 6)
5915 * field, compulsory fields: Mandatory Fields. (line 13)
5916 * field, forbidden fields: Prohibited Fields. (line 6)
5917 * field, mandatory fields: Mandatory Fields. (line 13)
5918 * field, special fields: Record Sets Properties.
5920 * floating point numbers: Scalar Field Types. (line 52)
5921 * foreign key: Other Field Types. (line 27)
5922 * foreign key <1>: Foreign Keys. (line 67)
5923 * formatted output: Invoking recfmt. (line 6)
5924 * fractions: Scalar Field Types. (line 52)
5925 * general date syntax: General date syntax. (line 6)
5926 * grouping: Grouping Records. (line 6)
5927 * grouping, within regular expressions: Regular Expressions. (line 42)
5928 * hexadecimal: Scalar Field Types. (line 13)
5929 * ID numbers: Auto-Generated Fields.
5931 * implies, logical implication: Arbitrary Constraints.
5933 * inserting new records: Invoking recins. (line 6)
5934 * integers: Scalar Field Types. (line 9)
5935 * integrity problems: Declaring Types. (line 52)
5936 * integrity problems <1>: Mandatory Fields. (line 15)
5937 * integrity problems <2>: Keys and Unique Fields.
5939 * integrity problems <3>: Arbitrary Constraints.
5941 * integrity problems <4>: Remote Descriptors. (line 35)
5942 * integrity problems <5>: Confidential Fields. (line 62)
5943 * integrity, checking: Checking Recfiles. (line 6)
5944 * integrity, checking <1>: Invoking recfix. (line 6)
5945 * interactive use: Bash Builtins. (line 6)
5946 * ISO 8601 date and time of day format: Combined date and time of day items.
5948 * ISO 8601 date format: Calendar date items. (line 30)
5949 * items in date strings: General date syntax. (line 6)
5950 * join: Joining Records. (line 65)
5951 * key, foreign key: Foreign Keys. (line 67)
5952 * key, primary key: Auto-Generated Fields.
5954 * language, in dates: General date syntax. (line 36)
5955 * language, in dates <1>: General date syntax. (line 40)
5956 * leap seconds: General date syntax. (line 65)
5957 * leap seconds <1>: Time of day items. (line 14)
5958 * leap seconds <2>: Seconds since the Epoch.
5960 * license, GNU Free Documentation License: GNU Free Documentation License.
5962 * literals, numeric literals: SEX Operands. (line 12)
5963 * literals, string literals: SEX Operands. (line 30)
5964 * locale: Scalar Field Types. (line 54)
5965 * locale <1>: Date and Time Types. (line 11)
5966 * looking up data: Selecting by predicate.
5968 * MacKenzie, David: Authors of parse_datetime.
5970 * mandatory fields: Record Sets Properties.
5972 * mandatory fields <1>: Mandatory Fields. (line 6)
5973 * mdb: Invoking mdb2rec. (line 6)
5974 * mdb2rec: Invoking mdb2rec. (line 6)
5975 * Meyering, Jim: Authors of parse_datetime.
5977 * minutes, time zone correction by: Time of day items. (line 29)
5978 * month names in date strings: Calendar date items. (line 38)
5979 * months, written-out: General date syntax. (line 32)
5980 * MS Access: Invoking mdb2rec. (line 6)
5981 * multiline field values: Fields. (line 37)
5982 * multiline field values <1>: String Field Types. (line 14)
5983 * mutating field values: Setting Fields. (line 6)
5984 * numbers, written-out: General date syntax. (line 22)
5985 * octal: Scalar Field Types. (line 13)
5986 * operands, SEX operands: SEX Operands. (line 6)
5987 * operators: Size Constraints. (line 23)
5988 * operators, arithmetic operators: SEX Operators. (line 13)
5989 * operators, boolean operators: SEX Operators. (line 23)
5990 * operators, comparison operators: SEX Operators. (line 36)
5991 * operators, conditional operator: SEX Operators. (line 76)
5992 * operators, in selection expressions: SEX Operators. (line 6)
5993 * operators, string operators: SEX Operators. (line 68)
5994 * order of fields: Sorted Output. (line 58)
5995 * ordinal numbers: General date syntax. (line 22)
5996 * parentheses, in selection expressions.: SEX Operands. (line 101)
5997 * passwords: Confidential Fields. (line 6)
5998 * Pinard, F.: Authors of parse_datetime.
6000 * primary key: Keys and Unique Fields.
6002 * primary key <1>: Auto-Generated Fields.
6004 * prohibited fields: Prohibited Fields. (line 6)
6005 * pure numbers in date strings: Pure numbers in date strings.
6007 * quotation marks: Selecting by predicate.
6009 * quotation marks <1>: SEX Operands. (line 42)
6010 * range, type description: Declaring Types. (line 12)
6011 * ranges: Scalar Field Types. (line 25)
6012 * readability: Purpose. (line 35)
6013 * readability <1>: Foreign Keys. (line 70)
6014 * reals: Scalar Field Types. (line 52)
6015 * rec, type description: Foreign Keys. (line 67)
6016 * rec2csv: Invoking rec2csv. (line 6)
6017 * recdel: Invoking recdel. (line 6)
6018 * recfix: Syntactical Errors. (line 14)
6019 * recfix <1>: Invoking recfix. (line 6)
6020 * recfmt: Generating Reports. (line 32)
6021 * recfmt <1>: Invoking recfmt. (line 6)
6022 * recinf: Invoking recinf. (line 6)
6023 * recins: Invoking recins. (line 6)
6024 * record: Records. (line 6)
6025 * record sets: Record Sets. (line 6)
6026 * record sets <1>: Foreign Keys. (line 6)
6027 * record size: Records. (line 20)
6028 * record size <1>: Size Constraints. (line 6)
6029 * recsel: Selecting by predicate.
6031 * recsel <1>: Invoking recsel. (line 6)
6032 * recset: Invoking recset. (line 6)
6033 * regexp, type description: String Field Types. (line 33)
6034 * regular expressions: Regular Expressions. (line 6)
6035 * relative items in date strings: Relative items in date strings.
6037 * remote descriptors: Remote Descriptors. (line 53)
6038 * renaming fields: Renaming Fields. (line 6)
6039 * reports: Generating Reports. (line 6)
6040 * requiring certain fields in records: Mandatory Fields. (line 6)
6041 * restricting fields from records: Prohibited Fields. (line 6)
6042 * restricting fields from records <1>: Allowed Fields. (line 6)
6043 * restricting values of fields: String Field Types. (line 33)
6044 * restricting values of fields <1>: Arbitrary Constraints.
6046 * retrieving data: Selecting by predicate.
6048 * Salz, Rich: Authors of parse_datetime.
6050 * selecting records: Selecting by predicate.
6052 * selecting records <1>: Invoking recsel. (line 6)
6053 * selection expressions: Selection Expressions.
6055 * selection expressions <1>: Selecting by predicate.
6057 * shell: Bash Builtins. (line 6)
6058 * size, field size: String Field Types. (line 19)
6059 * size, record size: Records. (line 20)
6060 * size, record size <1>: Size Constraints. (line 6)
6061 * size, type description: String Field Types. (line 19)
6062 * sorting: Sorted Output. (line 6)
6063 * sorting <1>: Sorting Records. (line 6)
6064 * sorting <2>: Invoking recsel. (line 40)
6065 * sorting <3>: Invoking recfix. (line 30)
6066 * sorting, physically: Sorting Records. (line 6)
6067 * special fields: Record Sets Properties.
6069 * special fields <1>: Semantic Errors. (line 6)
6070 * special fields, list of: Record Sets Properties.
6072 * spots: Generating Reports. (line 55)
6073 * string operators: SEX Operators. (line 68)
6074 * strings: String Field Types. (line 6)
6075 * subscripts, in selection expressions: SEX Operands. (line 79)
6076 * templates: Generating Reports. (line 32)
6077 * templates <1>: Templates. (line 6)
6078 * time of day item: Time of day items. (line 6)
6079 * time zone correction: Date and Time Types. (line 11)
6080 * time zone correction <1>: Time of day items. (line 29)
6081 * time zone item: General date syntax. (line 40)
6082 * time zone item <1>: Time zone items. (line 6)
6083 * time, fields containing time values: Date and Time Types. (line 6)
6084 * timestamps: Time-Stamps. (line 6)
6085 * types: Types and Fields. (line 6)
6086 * unique fields: Keys and Unique Fields.
6088 * unique identifiers: Unique Identifiers. (line 6)
6089 * URL: Remote Descriptors. (line 53)
6090 * UUID: Other Field Types. (line 16)
6091 * uuid: Unique Identifiers. (line 6)
6097 Node: Introduction
\x7f7569
6098 Node: Purpose
\x7f7793
6099 Ref: Purpose-Footnote-1
\x7f10820
6100 Node: A Little Example
\x7f10856
6101 Node: The Rec Format
\x7f13522
6102 Node: Fields
\x7f14124
6103 Node: Records
\x7f15906
6104 Node: Comments
\x7f16947
6105 Node: Record Descriptors
\x7f18338
6106 Node: Record Sets
\x7f18950
6107 Node: Naming Record Types
\x7f21795
6108 Node: Documenting Records
\x7f22781
6109 Node: Record Sets Properties
\x7f24155
6110 Node: Querying Recfiles
\x7f26455
6111 Node: Simple Selections
\x7f27521
6112 Node: Selecting by Type
\x7f31623
6113 Node: Selecting by Position
\x7f33547
6114 Node: Random Records
\x7f35284
6115 Node: Selection Expressions
\x7f36540
6116 Node: Selecting by predicate
\x7f37477
6117 Node: SEX Operands
\x7f41115
6118 Node: SEX Operators
\x7f43958
6119 Node: SEX Evaluation
\x7f46725
6120 Node: Field Expressions
\x7f48229
6121 Node: Sorted Output
\x7f50760
6122 Node: Editing Records
\x7f54574
6123 Node: Inserting Records
\x7f55439
6124 Node: Adding Records With recins
\x7f56790
6125 Node: Replacing Records With recins
\x7f58744
6126 Node: Adding Anonymous Records
\x7f59826
6127 Node: Deleting Records
\x7f61144
6128 Ref: Deleting Records-Footnote-1
\x7f62755
6129 Node: Sorting Records
\x7f62919
6130 Node: Editing Fields
\x7f63788
6131 Node: Adding Fields
\x7f64881
6132 Node: Setting Fields
\x7f66073
6133 Node: Deleting Fields
\x7f67072
6134 Node: Renaming Fields
\x7f67819
6135 Node: Field Types
\x7f68334
6136 Node: Declaring Types
\x7f69796
6137 Node: Types and Fields
\x7f71887
6138 Node: Scalar Field Types
\x7f72933
6139 Node: String Field Types
\x7f75088
6140 Node: Enumerated Field Types
\x7f77028
6141 Node: Date and Time Types
\x7f78378
6142 Node: Other Field Types
\x7f79362
6143 Node: Constraints on Record Sets
\x7f80689
6144 Node: Mandatory Fields
\x7f81876
6145 Node: Prohibited Fields
\x7f83778
6146 Node: Allowed Fields
\x7f85186
6147 Node: Keys and Unique Fields
\x7f86014
6148 Node: Size Constraints
\x7f88414
6149 Node: Arbitrary Constraints
\x7f89559
6150 Node: Checking Recfiles
\x7f91433
6151 Node: Syntactical Errors
\x7f92125
6152 Node: Semantic Errors
\x7f92999
6153 Node: Remote Descriptors
\x7f94278
6154 Node: Grouping and Aggregates
\x7f97240
6155 Node: Grouping Records
\x7f97689
6156 Node: Aggregate Functions
\x7f100499
6157 Node: Queries which Join Records
\x7f104649
6158 Node: Foreign Keys
\x7f106698
6159 Node: Joining Records
\x7f108987
6160 Node: Auto-Generated Fields
\x7f112264
6161 Node: Counters
\x7f115287
6162 Node: Unique Identifiers
\x7f116346
6163 Node: Time-Stamps
\x7f117644
6164 Node: Encryption
\x7f118113
6165 Node: Confidential Fields
\x7f119086
6166 Node: Encrypting Files
\x7f122565
6167 Node: Decrypting Data
\x7f123624
6168 Node: Generating Reports
\x7f126337
6169 Node: Templates
\x7f128875
6170 Node: Interoperability
\x7f130223
6171 Node: CSV Files
\x7f130740
6172 Node: Importing MDB Files
\x7f132920
6173 Node: Bash Builtins
\x7f135226
6174 Node: readrec
\x7f136637
6175 Node: Invoking the Utilities
\x7f139254
6176 Ref: Common Options
\x7f139570
6177 Node: Invoking recinf
\x7f140479
6178 Node: Invoking recsel
\x7f141793
6179 Node: Invoking recins
\x7f146108
6180 Node: Invoking recdel
\x7f149151
6181 Node: Invoking recset
\x7f151119
6182 Node: Invoking recfix
\x7f153938
6183 Node: Invoking recfmt
\x7f156030
6184 Node: Invoking csv2rec
\x7f156642
6185 Node: Invoking rec2csv
\x7f157360
6186 Node: Invoking mdb2rec
\x7f158501
6187 Node: Regular Expressions
\x7f159268
6188 Node: Date input formats
\x7f161519
6189 Node: General date syntax
\x7f163956
6190 Node: Calendar date items
\x7f166940
6191 Node: Time of day items
\x7f168944
6192 Node: Time zone items
\x7f171147
6193 Node: Combined date and time of day items
\x7f172404
6194 Node: Day of week items
\x7f173266
6195 Node: Relative items in date strings
\x7f174281
6196 Node: Pure numbers in date strings
\x7f177090
6197 Node: Seconds since the Epoch
\x7f178078
6198 Node: Specifying time zone rules
\x7f179705
6199 Node: Authors of parse_datetime
\x7f182085
6200 Ref: Authors of get_date
\x7f182271
6201 Node: GNU Free Documentation License
\x7f183234
6202 Node: Concept Index
\x7f208397