Text, cBase, and Dive Input Build Tags

• gregorian month—Gregorian year-month
• gregorian quarter—Gregorian year-quarter
• iso8601 week—ISO-8601 year-week
• gregorian+<N> month—Offset Gregorian year-month for fiscal calendars. Specifying "gregorian+6 month" means "add six months to the real year-mo to get the fiscal year-mo", which is like saying "The fiscal year starts in July".
• <custom calendar file>—For more tailored fiscal calendars

• See Spectre Calendars.

Defines the columns for the cBase when using text-input.

• If column headings exist, Spectre attempts to auto-detect the type; however, it is a good idea to check these and correct as necessary to ensure faster processing and less memory usage.
• If column headings do not exist, you must specify all columns for Spectre to know the names. The empty string cannot be used as a column name. Column names are case insensitive.
• If no columns are specified, all are brought in.
• It is a best practice to specify which columns to keep rather than which to drop.
• If a column is declared but not found in the input, the build fails. If multiple inputs are in use and some of them include the column, then the build succeeds by assigning null values.

Column tags are often followed by type, calendar, format, duration, or input-format options to clarify the data being read and added to the cBase. Specifying the type is a best practice.

When used in a lookup block, column supports the suggested-dimension tag. This allows you to flag the additional column as an optional dimension, avoiding the dimcount process used for ProDiver's console display.

NOTE: The column tag cannot be used with cbase-input, but it can be used with tunnel-input.

Specifies the delimiter for the flat file. If delimiter is not specified, Spectre attempts to auto-detect the value. Best practice is to specify the delimiter value. The value must be a string and is either a single character (for example, "\t" for tab, "," for comma, " " for space, ";" for semicolon, "|" for pipe) or the special strings "tab" or "comma".

Specifies the path to a dictionary file to use with text-input. Dictionary files are used to specify column names when the input lacks column headers. A sample dic file:

version "1";

// Computer generated object language file

object 'DICT' "Main" {

type="variable",

delimiter=" ",

columns={

{ Name="Colour" },

{ Name="Taste" },

{ Name="Weight" },

{ Name="Value" }

}

};

NOTE: Dictionary files are considered legacy functionality—not all options are supported.

Drops the named column from the input file when creating the cBase.

When using text-input, if you want to specify only which columns to keep, so that the build drops all others, use the column tag to specify the columns.

Keep in mind that dropping columns only affects the detail file size and build time, it does not affect the performance of dives.

Defines the encoding used in the input and output files. Encoding types supported include: cp-1252, iso-8859-1, utf-16be, utf-8, and gb18030.

NOTE

• Best practice is to specify the encoding of the input data.
• For text inputs, Spectre analyzes the first few megabytes of the file to attempt to determine the encoding.
• For databases, see ODBC Input Build Tags.
• Spectre Build's informational output indicates which encoding is in use.
• Starting with version 27, any incoming bytes that are not valid per the specified or assumed encoding are replaced with placeholders; these appear as squares in ProDiver.
• Non-Unicode DI clients are Unicode-aware.
• See also Integrator Unicode Data Support.

Sets both the display format and the input properties, unless a separate input-format is specified.

Formats the 'date' or 'period' data for the cBase column.

• To store dates like July 4, 2013, use format="MMMM D, YYYY".
• To include letters such as FQ (fiscal quarter), as in YYYY-FQQQ, backslashes are needed to force the characters to be taken literally, and another backslash is needed to escape each backslash in the string.

Formats numeric data, for example as currency.

Also indicates how to format Boolean data in the cBase column.

• The default format is "True;False".
• Other alternatives are "T;F", "Yes;No", and "true".
• If the format has a blank after the semicolon, or if there is no semicolon, then blank values are considered false. Otherwise, when there is a semicolon and the string after it is not empty, blank values are parsed as null.

Formats the datetime data.

• To indicate the data is zoned time and should be stored as GMT, use the zone option.
• To specify the default time zone for display, use the time-zone tag.

See also: Spectre Format Conventions.

Clarifies the incoming data.

• For dates and periods, the default format is YYYY/MM/DD. Use input-format to define the date format in the input file to ensure correct interpretation of date data.
• For datetime data with time zones, use input-format with the zone option to save the data in the cBase in GMT. Use the time-zone option to specify the default display. SeeSpectre Format Conventions.
• For Boolean data, use input-format to define the data as it appears in the input file.

Also see format.

Specifies an inline list as the data source. For details on how to enter the list data, see About Spectre List Input.

Options are the same as for text-input, except you cannot specify encoding, headers (always true), or filename-column.

The default delimiter is a space rather than a tab. You can specify an alternate delimiter or column type.

Specifies a system property for the column.

Other system properties defined using the property tag include:

• Definition

• Reference

• Good Direction

See Spectre Properties.

Specifies an alternate sort. When specifying columns, use sort-by to order them based on another column's value.

NOTE

• An alternate sort can only be applied to a string column, and the sort-by column must be a string or integer column.
• You cannot add, remove, or change alternate sorts after a cBase is built.
• The sort-by column does not need to be built into the cBase.
• Dive and cPlan files also have a sort tag, but for string data, rely on alternate sorts established (if any) when the cBase is built. Dives can also specify sorting on numeric columns.

• Spectre uses "numeric collation" when sorting strings. When a subset of a string looks like a number, then the numeric value is considered during the sort. For example, Spectre sorts "a2" before "a10". See also sort-natural.

• Null strings sort at the end.

• If two distinct strings in the column (to be sorted) are set to sort as the same value in the sort-by column, then the normal ordering of those distinct strings will resolve the tie.

• If a row has the value null or the value unknown in the column with an alternate sort, then the sort-by column must also have the same value.

• If the sort-by column has type string AND has the value null or the value unknown, then the column to be sorted must have the same value.

• Spectre issues a warning if you attempt to alternate sort null as non-null, or unknown as not-unknown.

Allows you to promote (true) or demote (false) a column.

NOTE

• Set to true to promote a numeric column type to become a diveable dimension and have it appear in the ProDiver console.
• Set to false to demote a column from classification as a dimension so it does not appear in the ProDiver console.
• When set to false, the new dimension does not appear in the console (which requires dimcount processing), but it can still be promoted to a dimension for diving—equivalent to the concept of a dynamic dimension for a classic model.
• Using required-dimension on a column effectively sets suggested-dimension=false.

• Wildcard characters ("*" and "?") are supported.
• Zip files are supported, and the files within are treated as multiple files.
• Windows has a limit of 16,384 files. On Linux, the limit is set by an administrator.
• A build block can include both text-input and cbase-input tags.
• Columns from all inputs go into the cBase. Columns in secondary inputs need to be explicitly declared to be included. If an input does not have a column, the cBase column is created with nulls. If nulls are not appropriate, an add operation can be used under the input to specify the missing values.
• Operations outside the input block operate on the union of the inputs for the build.
• If multiple text files are listed using a single text-input tag, there is an option to parse the files in parallel when building the cBase. (See Spectre build Options and Build cBase Process Node.)

• When there is no child block after text-input, all columns are used and their types are inferred. Use of a child block allows you to limit columns and apply build operations. See the table for Build Operations Tags under General Build Tags.

TIP: The input files can use a wildcard character in the filename, and as of 7.1(13), in the directory name as well.

Specifies the type of data in the column.

• Valid values are boolean, date, datetime, double, fixed100, integer, period, and string. See Spectre Data Types.
• If the type is "period", a calendar tag is required. Here in the final snippet (deprecated), the format tag says to use the standard Gregorian counterparts—for example: 2016-01 (2015-Jul).
• If the type is datetime, a zone tag can be included as part of the format to indicate a specific time zone.
• The type for date assumes that the input has a formatted date, not the binary date value.