MagIC Help Library -- MagIC Console Software

3 MagIC Console Software

Getting your data organized, complete and ready to submit for a paper publication or ingesting into a database is a time-consuming job. We have developed the MagIC Console Software to aid you in your collation of paleomagnetic and rock magnetic data and to make the data ready for uploading in the MagIC database.

3.1 Introduction to the Standardized MagIC Data Format

Establishing an integrated paleomagnetic and rock magnetic database is difficult as we need to move large quantities of data into the MagIC database from both legacy and new studies. To address this challenge we have chosen an approach that makes the archiving of these data Synchronous with the Scientific Publication process and is entirely based on User Contributions. Around the time of publication each scientist has all relevant data for a publication at his fingertips and knows best how to deal with it. In fact, he probably went through a sustained effort to collect all measurement data (and most of the relevant metadata) to perform his or her scientific research. The MagIC Console Software is developed to aid the scientist to collate all information at this opportune time and to do so by making use of a Standardized MagIC Data Format.

3.1.1 What is a SmartBook?

We wish to tap into this process by supplying scientists with a tool that they can use to collect all data that are relevant to one particular publication (henceforth referred to as a project). Key to this approach is a standard Data and Metadata Template in the form of an Microsoft Excel^© SmartBook in which they can enter their data and in which they can further process their data to eventually upload into the online MagIC database.

Because we use this standardized SmartBook protocols can be established around which scientists can build (or adapt) their (current) laboratory protocols. For example, they can streamline the collection of measurement data by enabling the export of standard MagIC Format Text Files that can be readily imported into the MagIC SmartBooks. Similar export functions can be established for various data reduction software and geomagnetic modeling codes. This approach significantly increases the flow of magnetic data into the MagIC database and circumvents labor-intensive data entry for individual scientists.

The MagIC SmartBook has been defined in a way so that it can store all measurements and their derived properties for studies of paleomagnetic directions and intensities, and rock magnetic experiments, such as hysteresis, remanence, susceptibility and anisotropy. The basic design of this SmartBook focuses on the work-flow in typical paleomagnetic and rock magnetic studies. This ensures that individual data points can be traced between actual measurements and their related specimens, samples, sites, locations, and so forth.

To make working with the MagIC SmartBooks more straightforward, we have developed the MagIC Console Software that, in addition to many other functions, contains an Import function for standard text files. This software helps you to Enter Data, it helps you to check the Correctness and Coherence of the data entries, and it helps you Prepare for Uploading your SmartBook in the MagIC database. Since the MagIC SmartBooks have been developed as Microsoft Excel^© Files, most users will feel comfortable with the setup, which makes use of standard toolbars and dialogboxes.

3.1.2 Structure of the MagIC SmartBooks

The MagIC SmartBooks are standard Microsoft Excel^© Workbooks that contain predefined tables stored in separate worksheets. There is an obvious hierarchy in these tables, where the EarthRef (ER) tables are the most general and are applied to other databases hosted under this umbrella website as well.

Note For a detailed overview please open the MagIC.vXX.metadata.definition.xls file in Microsoft Excel^© and use the hyperlinks in the Frontpage (or the standard Microsoft Excel^© tabs at the bottom of the page) to show the description for each MagIC table, including example input data. This information can also be found by following the http://earthref.org/MAGIC/metadata.htm web link or by selecting the Data and Metadata Definition option from the MagIC Help menu.

The four MagIC tables (measurements, methods, instruments and calibrations) are less general, but contain data and metadata that are common for typical paleomagnetic and rock magnetic projects. The PMAG and RMAG tables contain the most specialized, highly derived data.

3.1.3 MagIC Table Layout

Each table is divided into several pre-titled columns. The first Five rows of each column characterize the data to be entered below the headings. The first row gives the Label of the column and describes the data to be entered (explanation in plain English). The second row displays the Oracle Variable Name associated to these data in the relational database and used by the MagIC Console Software. The third row indicates the Data Type and maximum length of text strings. The fourth row shows the expected Unit of the data. The fifth row indicate the Status of the data to show whether the data is required, recommended or optional. All five heading rows are fixed in the MagIC Data and Metadata definition, which is strictly maintained and versioned.

Upon opening a SmartBook in the MagIC Console Software they will be used to check the validity of the tables and columns residing in the Microsoft Excel^© File.

3.1.4 Standard MagIC Text File Format

The MagIC Console Software has the ability to export and import Flat Text Files. To be able to easily transfer data between outside software packages, data collection computers and the MagIC SmartBooks, we have developed the Standard MagIC Text File Format. Experience shows that this is the most efficient way when dealing with large data sets. It allows you to quickly import these data into the MagIC SmartBooks without making mistakes. In the example below, we show how this text file is based on the layout of the table displayed above.

Header Structure

The Standard MagIC Text File Format starts out with two header lines (red fonts) indicating the delimiter used, table name and column names. Below these header lines, the data appears in the same order as indicated by the column names (black fonts).

Multiple Data Blocks

Possible delimiters include tab and pipe symbols. Note that single or double quotes around text strings are not required. You can also store multiple data blocks (for one project) in one text file, where each data block is stored according to the above rules but is separated by the standard >>>>>>>>>> divider. Since each block has its own header lines, in principle, you can store the results from different experiments or tables in one and the same Standard MagIC Text File. This will result in a text file that may look as follows.

Note Because in the above example both experiments KOPA-2004-01 and KOPA-2004-05 are properly divided and because each block has its own header lines, each block can store different parameters and in a random order (compare the placement of the treatment_temp field, for example). This will keep the file size minimal because only the data relevant for that one particular experiment needs to be stored. It is also flexible, because it allows you to store data measured using different equipment or measurement protocols in one-and-the-same data file.

Always store the text files in the same directory as the MagIC SmartBook file itself.

3.1.5 Software Installation

The MagIC Console Software does not work with a fixed directory structure. However, we recommend that you generate a main MagIC directory on your hard disk and that you store the populated SmartBook for each of your MagIC Projects in sub directories.

Required Files

In the table below we have listed Four files that are required for a proper functioning of the MagIC Console Software. Note that each file has been named according to its current version, that is v20 or v24 (instead of vXX) for example.

Note Every time you download the software from the MagIC Website you will receive a zipped archive that contains all four files (see table) and some subdirectories with example SmartBook files. When unzipping this software archive, be sure to retain the relative paths and folder names.

3.2 Entering New Data

While populating the MagIC SmartBooks there are several possible approaches to put your paleomagnetic and rock magnetic data into these Microsoft Excel^© Files. Here we will provide a guided explanation on how to populate the MagIC SmartBooks most effectively, using a top-down approach, where first the high-level data and metadata are entered, followed by more detailed magnetic measurement and derived data later on.

Note that this an example only that allows you to get familiar with the capabilities of and functions available in the MagIC Console Software. There may be better ways to go about this for certain special cases or data sets, as described elsewhere in this help library.

3.2.1 Overview of the Data Population Process

We start out by giving an overview of the MagIC Data Population process, which can be divided into four different parts. We will follow this general overview by a Step-by-Step Guided Explanation of the process.

Starting a New MagIC SmartBook

Start the MagIC Console Software within Microsoft Excel^©
Generate a New SmartBook
Run the MagIC Wizard to adapt the SmartBook for your current data project
Add your current Data Project Citation in the ER_citations table
Compile a specific list of Methods for your project
Compile a specific list of Instruments for your project

Entering the High Level Metadata

Add the Citations of other studies in the ER_citations table for proper attribution to your data
Add the Mail Addresses of the analysts, rock archivers and scientists to the ER_mailinglist table
Define the Data and Metadata for expeditions, localities, sites, samples, specimens, minerals, fossils and synthetic materials in the general EarthRef.org [ER] tables
Add Age Data to the ER_ages table

Entering and Importing PMAG and RMAG Data

Add Measurement Data in the MAGIC_measurements table
Add Paleomagnetic and Rock Magnetic Data in the PMAG and RMAG tables

Preparation for Uploading

Perform Data Checks from the Special menu
Perform the Prepare for Uploading function
Upload the XLS and TXT versions of your SmartBook at http://earthref.org/MAGIC/upload.htm

The above 15 steps will be discussed in detail in the Step-by-Step Guided Explanation in the next help topic. Remember, however, that this is a generalized approach. Some steps may not be required (and thus can be skipped) because they are not relevant to your project. Therefore, it is especially important to always run the MagIC Wizard at the beginning of your data population session.

3.2.2 Step-by-Step Guided Explanation

1. Start the MagIC Console Software

In Microsoft Excel^© open MagIC.vXX.console.xls via the File # Open menu.
Enable Macros when prompted.
A new MagIC Toolbar will appear in the Microsoft Excel^© window.

Note On Macintosh^© computers, in particular, you may have to first adjust the Security settings under the Tools menu. Set the Security Level to Medium so you can choose whether or not to enable macros. If you set the security level to low, you will not be asked to enable macros when opening the software (not recommended).

If you encounter an Object Library Not Registered error message on Mac OS X systems, your Microsoft Office^© installation contains a corrupt library/preferences/Microsoft/Office Registration Cache X file. Open this file in a text editor, where you should remove all text and save it. Finish by locking this file and restarting Microsoft Office^©. The locking of this text file is most essential, otherwise the problem will re-occur.

Upon opening the MagIC Console Software a check will be performed for Software Updates of the software itself and the Controlled Vocabularies file. This check can only be carry out if you are Online with your computer. If an update is available, a message will appear guiding you to the Software Download web page.

2. Generate a New SmartBook File

From the MagIC Toolbar select Operations # New SmartBook.
Browse for your folder to store the SmartBook file, supply a name for your new file, and finish by clicking the Save button.

Note You will note that after creating a new file (or opening an existing file) that three checks will be performed on your MagIC SmartBook file. First the Version of your SmartBook will be checked. If your file has an older version number, the software will automatically update your file to the latest MagIC SmartBook definition. Then the Integrity of your file will be checked to assure that all tables and columns are present and spelled correctly. This check guarantees that unintentional changes to the SmartBook file are detected and repaired. Finally, the Format of your file is checked and adjusted, if necessary.

3. Run the MagIC Wizard

On the MagIC Toolbar click Wizard.
Check and Uncheck the checkboxes in each wizard step, and finish by clicking the Finish button.

Note The MagIC Wizard tool will temporarily hide tables and columns from view, depending on the options you selected. As a result, you can customize your MagIC SmartBook for your own needs. For example, if you only have performed Paleomagnetic Analyses, you can uncheck the Rock Magnetics checkbox in the first step of the wizard, which effectively hides all the RMAG tables and data columns specific to rock magnetic experiments in the MAGIC_measurements table. But you can also individually hide or view rock magnetic experiments (susceptibility, remanence, hysteresis, FORC and anisotropy) in the second step of this wizard.

Note that the selections made in the MagIC Wizard are only applied after clicking the Finish button. You can show all tables and data columns again by clicking the Show All button in the wizard, followed by clicking the Finish button. Although we recommend that you run the MagIC Wizard at the beginning of data population, you can always re-run this wizard while further adapting the settings.

4. Add Your Current Project Citation in the ER_citations Table (This Study)

From the MagIC Toolbar select Tables # ER_citations.
On the MagIC Toolbar click Edit Data.
Click the New Record button and fill out the reference to your project. Use the Hot Buttons on the right-hand side of this dialogbox to easily add Author Names and Keywords, and to have access to shortlists with Journal Names and Citation Types.
When finished, hit the Close button to exit this dialogbox. Your citation will be placed in the first row of the ER_citations table (see below) and is designated as This study.

5. Compile Methods Definitions for this Project

From the MagIC Toolbar select Special # Add and Remove Method Definitions.
Browse through all Method Categories using the buttons in the upper part of this dialogbox and Add the appropriate Method Codes for your project to the In Use ... list.
When finished compiling the Method Codes for your project, click on the Save button to close this dialogbox and to add these codes to the MAGIC_methods table. Note that references associated with these Method Codes will be automatically stored in the ER_citations table.

Note If you cannot find the appropriate Method Code by using this dialogbox, you can always request a New Method Code from the MagIC Database Team. If a new code is warranted, it will be added to the master listing of methods that is part of the Controlled Vocabularies file.

You can find a complete listing of Method Codes at the http://earthref.org/MAGIC/methods.htm web-page. You can also reach this page by selecting Method Codes in the MagIC Help menu.

Although we recommend that you run the Add and Remove Method Definitions at the beginning of data population, you can always call up this dialogbox and further adapt the list of In Use ... codes. This means that you can also remove Method Codes that later proof to be not needed in your MagIC SmartBook.

6. Compile Instrument Definitions for this Project

From the MagIC Toolbar select Special # Add and Remove Instrument Definitions.
Browse for the instruments and Add the appropriate Instrument Codes for your project to the In Use ... list. You can also search for specific instruments by typing in their (partial) name in the textbox on top of this dialogbox, followed by clicking the Find ... button.
When finished compiling the Instrument Codes for your project, click on the Save button to close this dialogbox and to add these codes to the MAGIC_instruments table. Note that references associated with these Instrument Codes will be automatically stored in the ER_citations table.

Note If you cannot find the appropriate Instrument Code by using this dialogbox, you can always define a New Instrument Code by activating the MAGIC_instruments table and clicking the Edit Data button on the MagIC Toolbar. Continue by clicking the New Record button in order to generate a new record and fill out all fields. After uploading your MagIC SmartBook into the MagIC Database this new code will be reviewed and (if a new code is warranted) it will be added to the master listing of instruments.

You can find a complete listing of Instruments Codes at http://earthref.org/MAGIC/instruments.htm. You can also reach this page by selecting Instrument Codes in the MagIC Help menu.

Although we recommend that you run Add and Remove Instrument Definitions at the beginning of data population, you can always call up this dialogbox and further adapt the list of In Use ... codes. This means that you can also remove Instrument Codes that later proof to be unnecessary in your SmartBook.

7. Add other Citations to ER_citations table

From the MagIC Toolbar select Tables # ER_citations.
On the MagIC Toolbar click Edit Data.
Click the New Record button and fill out the reference. Use the Hot Buttons on the right-hand side of this dialogbox to easily add the Author Names and Keywords, and to have access to shortlists with Journal Names and Citation Types. Repeat this step until you've added all references.
When finished, click the Close button to exit this dialogbox. The references will be added to the bottom of the ER_citations table.

Note Only add references for publications from which you have been compiling additional data, or which you're referring to in order to describe expeditions, locations, sites, samples, specimens, etc. References associated with certain methods or instruments do not have to be filled in by hand, they are automatically retrieved when you've been using the Add and Remove Method Definitions and Add and Remove Instruments Definitions functions (see above).

8. Add all Mail Addresses to ER_mailinglist Table

From the MagIC Toolbar select Tables # ER_mailinglist.
On the MagIC Toolbar click Edit Data.
Click the New Record button and fill out an address. Repeat this step until you've added all addresses.
When finished hit the Close button to exit this dialogbox. The addresses will be added to the bottom of the ER_mailinglist table.

Note In this table add addresses for Analysts, Rock Archivers and Scientists that have been associated with this project, either as principal investigators or as participants in carrying out the science.

9. Define the Data and Metadata in the General EarthRef [ER] Tables

From the MagIC Toolbar select the appropriate Tables # ER_expeditions, ER_locations, ER_sites, ER_samples, ER_specimens, ER_fossils, ER_minerals or ER_synthetics. Whenever possible start out in the same order as listed, going from more general to more detailed, because the Location Names will be required in the ER_sites and ER_samples tables, for example (and so forth).

On the MagIC Toolbar click Edit Data.
Click the New Record button and fill out the data and metadata. Use the Hot Buttons on the right-hand side of this dialogbox to easily add Keywords, and to have access to shortlists that are linked to other tables, such as for Expedition Names, Location Names or Site Names. Repeat this step until you've added all data and metadata. You can also make use of the many Controlled Vocabularies that will pop up when clicking on the Green Hot Buttons.
When finished hit the Close button to exit this dialogbox.

Note Of course, you can type the data directly into the Microsoft Excel^© tables, and use standard functionality to Copy-and-Paste data within the current MagIC SmartBook or from other Microsoft Excel^© workbooks that are open in (a second instance of) Microsoft Excel^©. You can also use the Import ... option from the Operations menu to read in data from Standard MagIC Text Files.

When adding a new record, you can check the Prefill Records checkbox at the bottom of the Edit Data dialogbox. This setting will pre-fill your new record with the data from the "selected" data record in the current/active table. Use this functionality when adding data records that have the majority of their data field entries in common. This significantly improves the speed of manual data entry for larger data sets.

10. Add Age Data to the ER_ages Table

From the MagIC Toolbar select Tables # ER_ages.
On the MagIC Toolbar click Edit Data.
Enter the age data. Note that multiple age determinations for one and the same site, sample, mineral or fossil should be entered as separated records/rows in this MagIC SmartBook table.

Note This table has been designed to allow for Radiometric Ages and for age determinations that are related to Stratigraphy. For the latter category it is important to enter the information on the reference horizon (with a stratigraphic height of zero, by definition) as well, so that sample heights can be given relative to these horizons. Preferably, these reference horizons are GSSP's or otherwise internationally recognized strata.

Detailed Radiometric Age information is not required in this table. In the near future, however, such data can be deposited in the online Radio-Isotope Geochronology Databases that will reside under EarthChem.org. As an important result, both the EarthChem.org Geochronology and MagIC databases will be tightly linked with each other to allow for a seamless operation of these databases.

11. Add Measurement Data in the MAGIC_measurements table

From the MagIC Toolbar select Tables # MAGIC_measurements.
On the MagIC Toolbar click Edit Data, or use the Import function (see tip below).
Enter the measurement data. Take special note here of the measurement Units that all are in SI.

Note Due to the normally large volume of measurement data, it is advisable to make use of the Import function in the Special menu. To this effect, you should store your measurement data in the Standard MagIC Text Files. When using the Import function you can add the contents of these files to the MAGIC_measurements table, either as a single file or as multiple files in one batch.

12. Add Paleomagnetic and Rock Magnetic Data

From the MagIC Toolbar select the appropriate PMAG or RMAG tables.
On the MagIC Toolbar click Edit Data, or use the Import function (see tip below).
Enter the paleomagnetic data in the PMAG tables, starting with the Specimens data, followed by the Samples, Sites and Poles data. Finish by entering the rock magnetic data into the RMAG tables. Take special note here of the Units that all are in SI, unless otherwise indicated in the light blue header rows that are located on the top of each table.

Note Due to the normally large volume of derived data, it is advisable to make use of the Import function in the Special menu. To this effect, you should store your data in the Standard MagIC Text Files. When using the Import function you can add the contents of these files to the PMAG and RMAG tables, either as single files or as multiple files in one batch.

13. Perform Data Checks

From the MagIC Toolbar select Special # Check Data Records.
From the MagIC Toolbar select Special # Check Specific Data Records.
From the MagIC Toolbar select Special # Check for Data Integrity.

Note You can perform the checks for the Selected Table Only or for All Tables in the SmartBook.

You will be prompted to Edit Data for entries that generate errors. Click the Yes button, if you want to repair this entry on the fly. After you corrected the erroneous or missing data entries, these functions will continue to check the next records. If you click the No button, the data check will be aborted.

Before starting these data checks, you may want to run the Synchronize Names function under the Special menu. This function will go through a table and find links for which the definitions are missing in their host or parent tables. When missing links are found, they will be added to the parent tables, while appearing in highlighted "light blue" and "pink" cells for easy detection. The originating data records are highlighted in a "purple" color. You can remove the formatting again by pressing Ctrl Shift S or it will automatically be removed when you run the Prepare for Uploading function.

14. Prepare for Uploading

From the MagIC Toolbar select Operations # Prepare for Uploading.
Follow the instructions to Edit Data for entries that generate errors.

Note You will be asked for a confirmation. Click Yes to continue preparing the SmartBook for uploading. Click the No button to abort this action. If you proceed, five different Data Checks will first be performed before the data is saved into a single Tab Delimited ASCII File. This may take several minutes, depending on the complexity and the amount of data stored in the MagIC SmartBook file. Please be patient!

Only perform this action when you're confident that the entire SmartBook has been populated with the data and metadata for your project. If you want to perform any of the Data Checks beforehand, please use the Check Commands at the bottom of the Special menu (see previous step).

15. Upload your SmartBook into the MagIC Online Database

Go to the http://earthref.org/MAGIC/upload.htm webpage. You can do so by selecting the Operations # Upload Into MagIC Database menu item. This will open you browser with the correct web form.
Register yourself at EarthRef.org unless you already have a Log-in Name and Password.

Note If during the Uploading process you need some additional help, please click on the Help Tab on top of the online webform. This will launch the MagIC Online Help Library and gives you context sensitive help.

To register, click on the Register link in the Topmenu of the EarthRef.org website. If you have forgotten your password and/or username, please go to the http://earthref.org/databases/ERML/ webpage and click on the Forgot Password link. After filling in your email address, your registration info will be sent in an email message. Twice a year you will be sent a Reminder Email with all registration info and a simple link to edit your user profile.

3.3 Special Cases

There may be a few cases that require a special approach while populating the MagIC SmartBooks. In this Chapter we treat these special cases by briefly describing the issue and a possible solution.

3.3.1 When Requiring New Instrument Codes

If you cannot find the appropriate Instrument Code in the software, you can always create a new code by activating the MAGIC_instruments table and by clicking the Edit Data button on the MagIC Toolbar. Continue by clicking the New Record button and fill out all fields. After uploading your files this new code will be reviewed and, if this new code is acceptable, it will be added to the Controlled Vocabulary listing of instruments.

3.3.2 When Entering More Than 65,000 Measurement Data Records

The user cannot add more than 65,000 data records in a single table due to the limitations of the Microsoft Excel^© worksheets. However, there is one exception made for the records of the MAGIC_measurements table. In this table you can import as many measurements as required. If the software detects more than 65,000 records, it will automatically generate a table named MAGIC_measurements2, and so on.

3.3.3 When Starting Data Population from Measurement Files

It may be quite common that you start data population with a measurement file, which has been stored in the Standard MagIC Text File Format. When pursuing this route, first import all these text files into a SmartBook that have been prepared by applying the first six steps described of a typical file upload procedure. Follow this by running the Synchronize Names function from the Special menu to pre-fill the names of all Locations, Sites, Samples and Specimens (and others) throughout the tables of the SmartBook. Now you only have to complete these tables with some extra data and metadata.

When new records are added to these tables using the Synchronize Names function, they will appear in highlighted "light blue" and "pink" cells for easy detection. The originating data records are highlighted "purple" as well. You can remove the formatting again by pressing Ctrl Shift S or it will automatically be removed when you run the Prepare for Uploading function.

3.4 Examples

In this Chapter we will present examples of a few common tasks you most likely will perform during the uploading of your data. The examples are meant to give you a step-by-step visual summary of what you will have to do to complete these tasks.

3.4.1 Entering References in the ER_citations Table

In every SmartBook you have to enter the bibliographic information for your current contribution to the MagIC Database. You may also need to add references for publications from which you have been compiling additional data. Typically you would follow the next sequence of actions ...

From the MagIC Toolbar select Tables # ER_citations.
On the MagIC Toolbar click Edit Data and click the New Record button.

Click on the Dark Blue Hot Buttons on the right-hand side of this dialogbox to easily add Author Names using the Dynamic Authors Listbox. Type in the last name of every single author that appears on the publication in the textbox on the top, followed by his/her initials. Click on the Add button. When you have entered all Authors Names, click the Update button to apply the list to the long authors field and to automatically generate (the first part of) the citation name (excluding the year).

Note Always start the Last Name with a capital, whereas the Initials are (in principle) all in upper case. In the example above, you see all the possible formats that are allowed. If you try to enter an author name in any other format, the program will not allow you to add the name. If you have entered a valid author name, it will be appended to the bottom of the listbox.

Fill in the Year. Note that the citation name gets appended with the newly entered Year.
Use the Light Green Hot Buttons on the right-hand side of this dialogbox to have access to shortlists with Journal Names and Citation Types. Note that the journal names should not be abbreviated.

Fill Out the rest of the bibliographic information of the reference.
When finished, hit the Close button to exit the Edit Row Data dialogbox. Your citation will be placed at the bottom of the ER_citations table (see next page for an example). Click the Cancel button to ignore all edits.

Note The first entry in the ER_citations table always gets assigned This study as the short citation name. This is done on purpose to make it easier for you to link all your data in the MagIC SmartBook files to your own Reference. Because in more than 75% of the cases you will refer to your own study, it becomes easier to insert This study instead of the standard Crimson et al. 2005 citation. Be aware that if you replace the first entry with another citation, this new citation will be designated This study!

Only add references for publications from which you have been compiling additional data, or which you're referring to in order to describe expeditions, localities, sites, samples, specimens, etc. References associated with certain methods or instruments do not have to be filled in by hand, they are automatically retrieved when you've been using the Add and Remove Method Definitions and Add and Remove Instruments Definitions functions.

You may enter references that are In Review or In Press as your This Study reference. The way how the MagIC Database treats these references is different. An In Review reference is treated as unpublished data and cannot be activated during the upload process. An In Press reference is treated as a published paper and this contribution can actually be activated during the upload process. To indicate a reference as In Review or In Press you have to enter this information into the pages column instead of the actual page range. If some citation info is not known yet, then please enter Unknown or Not Specified in for example the title, volume, pages, doi, book editors, publisher and city columns.

3.5 Function by Function

3.5.1 Operations Menu

New SmartBook ...

To open a new MagIC SmartBook file, choose the New File ... command (or ctrl-N) in the Operations menu. In a dialogbox you will be asked to Save the SmartBook File under your name of choice. The NewFile.xls is a default name that you can overwrite, as necessary. When clicking on the Save button, a new and empty MagIC SmartBook is created with 30 predefined tables (worksheets).

Open Existing SmartBook ...

To open a MagIC SmartBook, choose the Open Existing File ... command (or ctrl-O) in the Operations menu. In a dialogbox you will be asked to browse and select the SmartBook file you would like to re-open in the console software.

Close and Close All ...

Select the Close ... or the Close All ... commands in the Operations menu to close the active MagIC SmartBook file(s). Upon closing the console software will ask you to save the file(s) first.

Save ...

Click the Save ... command (or ctrl-S) in the Operations menu to save the active MagIC SmartBook file. This command has the same functionality as the Save button on the left-hand side of the toolbar itself. Clicking Ctrl Shift and the Save button [Windows platform only] will save the SmartBook file after cleaning up and reformatting all tables.

Import Data Files ...

Click the Import Data Files ... command (or ctrl-M) in the Operations menu to import Standard MagIC Text Files with data to include in the SmartBook tables. An Import Data Files dialogbox appears in which you can browse for these standard text files. You can select multiple text files by holding down the Ctrl or Shift keys [Windows platform only]. When clicking on the Open button, the selected file(s) will be opened and the data will be imported into the MagIC SmartBook tables.

If the data in the Standard MagIC Text File to import don't have the expected format, an error message will appear. Note that data in a column with a misspelled Oracle Name will be ignored while importing the data. The software will give you an error message.

Note The Standard MagIC Text Files can be tab or pipe delimited and have a simple two header line structure followed by the body of data. The order of the data columns is not fixed and empty columns can be omitted.

If errors occur, an error log with the *.error.log extension is written and saved in the same directory as the SmartBook file.

Export

Click the Export command (or ctrl-E) in the Operations menu to export Standard MagIC Text Files. You will be asked to export data for the current (active) table only, or to export all tables in the entire MagIC SmartBook file. All data will be exported into a single tab delimited text file with your chosen file name.

Note Use the export and import functions to move large quantities of data between MagIC SmartBooks. First export the table into a Standard MagIC Text File, then switch to another opened SmartBook using the SmartBooks menu. Now select the Import Data Files ... command and import the just saved data again.

Batch Processing ...

To automatically Import data and immediately Prepare for Uploading (see above) choose the Batch Processing ... command from the Operations menu. You will be asked to select one or more Standard MagIC Text File(s) [multiple files are only allowed on the Windows platform] after which the data processing starts without any user interaction. If errors occur during the five Data Checks they will not cause a prompt, but instead they will be written to an Error Log. This is an efficient option if you prepare Standard MagIC Text Files outside of the MagIC Console Software and you only want to pass your data through the console to make them ready for uploading. Since all checks get performed without any interrupting for your entire data contribution, the error logs give you a good overview where you should improve on your data and metadata collection.

Prepare for Uploading

When all data have been entered in the MagIC SmartBook choose the Prepare for Uploading command (or ctrl-F) in the Operations menu. You will be asked for be asked for a confirmation. If you click the Yes button, five Data Checks will first be performed before the data is saved in a single tab delimited text file. This may take several minutes, depending on the complexity and the amount of data stored in the SmartBook. Clicking the No button will cancel this action.

The five Data Checks include a SmartBook integrity check, a check for orphaned data, a general data check, a specific data check (for longitudes, citation style, data ranges, etc.) and an integrity check for all related data fields. If one of these checks returns an error, the Prepare for Uploading command will ask you to edit (or add) certain data and metadata. If you do not edit the particular field at this stage, you will cancel the Prepare for Uploading action.

Upload Into MagIC Database

After the succesfull completion of the Prepare for Uploading command you are ready to upload your data on the MagIC Website. Goto http://earthref.org/MAGIC/upload.htm or click on the Upload Into MagIC Database command in the Operations menu.

3.5.2 Special Menu

Autofit Columns

The widths of all columns in a table may be automatically adjusted to Autofit the five header rows using the Autofit Columns command in the Special menu.

Clear Table

To clear all the data in a table (worksheet) choose the Clear Table command in the Special menu. You will be asked for a confirmation.

Add and Remove Method Definitions ...

Each MagIC Project only requires a limited number of Method Definitions. By clicking on the Add and Remove Method Definitions ... command in the Special menu you can generate a list of Method Codes that describe the methods used in your project, including field sampling techniques, lab protocols, parameter estimations, and more (see the buttons at the top of this dialogbox for a complete list of categories). The Method Codes that are currently stored in your MagIC SmartBook file are listed in the In Use ... list box shown on the right-hand side.

You can Add new Method Codes by first selecting a method from the master listing of methods list box on the left-hand side, followed by clicking the right-pointing Double Arrow button. Show different categories of the Method Codes in this master listing by clicking on the Category Buttons in the top of this dialogbox.

You can Remove the Method Codes one-by-one by selecting the code in the In Use ... list box, followed by clicking the other left-pointing Double Arrow button. You can remove all methods at once by clicking on the Clear Selection(s) button. The program will ask you for a confirmation.

When you are finished compiling your list of Method Codes, click the Save button. This action will store all newly assigned Method Codes in the MAGIC_methods table, and it will add the appropriate references to the ER_citations table.

Note If you cannot find the appropriate Method Code by using this dialogbox, you can always define a new Method Code by activating the MAGIC_methods table and clicking the Edit Data button on the MagIC Toolbar. Continue by clicking the New Record button in order to generate a new record and fill out all fields. After uploading your MagIC SmartBook files this new code will be reviewed and (if a new code is warranted) it will be added to the master listing of methods.

You can find a complete listing of Method Codes at http://earthref.org/MAGIC/methods.htm.

Add and Remove Instrument Definitions ...

Each MagIC Project only requires a limited number of Instrument Definitions. By clicking on the Add and Remove Instrument Definitions ... command in the Special menu you can generate a list of Instrument Codes that describes the instruments used in your project. The Instrument Codes that are currently stored in your MagIC SmartBook file are listed in the In Use ... list box shown on the right-hand side. The Instrument Codes should all start with an abbreviation indicating the host institution.

You can Add new Instruments by first selecting an instrument from the master listing of instruments list box on the left-hand side, followed by clicking the right-pointing Double Arrow button. You can search for Instruments by typing in (part of) a name in the top textbox, followed by clicking on the Find ... button.

You can Remove the Instruments one-by-one by selecting the code in the In Use ... list box, followed by clicking the left-pointing Double Arrow button. You can remove all instruments at once by clicking on the Clear Selection(s) button. The program will ask you for a confirmation.

When you are finished compiling your list of Instruments, click the Save button. This action will store all newly assigned Instrument Codes in the MAGIC_instruments table, and it will add the appropriate references to the ER_citations table.

Note If you cannot find the appropriate Instrument Code by using this dialogbox, you can always define a new Instrument Code by activating the MAGIC_instruments table and clicking the Edit Data button on the MagIC Toolbar. Continue by clicking the New Record button in order to generate a new record and fill out all fields. After uploading your MagIC SmartBook files this new code will be reviewed and (if a new code is warranted) it will be added to the master listing of instruments.

You can find a complete listing of Instruments Codes at http://earthref.org/MAGIC/instruments.htm.

Synchronize Names

Some names, such as Location, Site, Sample and Specimen names appear in more than one place in the MagIC SmartBooks. Use the Synchronize Names function in the Special menu to automatically pre-fill these names throughout all the tables. This requires that you have at least filled out these names in one of the SmartBook tables. When new records are added to these tables they will appear in highlighted "light blue", "pink" and "purple" cells for easy detection. You can remove the formatting again by pressing Ctrl Shift S or when you Prepare for Uploading.

Remove Duplicate Data Records

You can detect and remove Duplicate Data Records in your tables by choosing Remove Duplicate Data Records from the Special menu. It is recommended that you first Highlight duplicate records. If you select this option, the potential duplicates get highlighted with an "orange" color. If you opt to Remove the duplicates immediately, the duplicate record(s) will be deleted from you table instead, while the first instance will be retained. Note that this action cannot be undone.

Remove Empty Data Records

You can detect and remove Empty Data Records by choosing Remove Empty Data Records from the Special menu. You will be asked to remove empty data record for the current (active) table only, or to do so for all tables in the entire MagIC SmartBook file. Note that this action cannot be undone.

Combine Two Similar Data Records

In some cases, tables may have sets of data records that are very similar, where they for example have the same sample number, but one data record (i.e. row) contains inclination and declination data, and another one only paleointensity data. These data records could be combined (i.e. merged) into one record. In order to do so, select two similar data records in a table and choose Combine Two Similar Data Records from the Special menu. If there are no data conflicts, the rows will be combined on the first row and the second row gets deleted. If the data cannot be combined, an error message is displayed and the data conflicts are highlighted in "orange" colors. To select two rows in your table, push the control key and click on any cell to select the first and second row. Note that this action cannot be undone.

Check Data Records

To verify that the data in one table have the expected format, choose the Check Data Records command in the Special menu. You will be asked to check data for the current (active) table only, or to check all tables in the entire MagIC SmartBook file.

Check Specific Data Records

You can also verify whether specific data (like latitudes, longitudes, citations, magnetic moments, paleointensities, flags) have the expected format and range of values. Some automatic repairs will be performed in the background (e.g. recasting longitudes from the -180/180 to the 0/360 notation) during this data check, other checks may return error messages. To perform this function, choose the Check Specific Data Records command in the Special menu. You will be asked to check data for the current (active) table only, or to check all tables in the entire MagIC SmartBook file.

Check for Data Integrity

To validate whether data in one table are correctly related with other data tables in the SmartBook file, choose the Check Data Integrity command in the Special menu. You will be asked to check data for the current (active) table only, or to check all tables in the entire file. This action will search each table for existing relations and it will check them against their parent tables. If a relation is broken (i.e. the parent record does not exist, or you misspelled the daughter or parent record) then an error will occur. You will be asked to edit certain data and metadata.

3.5.3 Data Entry and Editing Dialogbox

There are various ways to enter your data in the MagIC SmartBooks. One of these methods is the Edit Data option (or Ctrl Shift D) that you can find on the main MagIC Toolbar. In this dialogbox you can enter, edit and delete data on a record-by-record basis. If you want to edit an existing data record, first select the data record you want to edit by selecting any cell in its row in the Microsoft Excel^© table, followed by clicking on the Edit Data button. You can also Double-Click on the cell. The complete record will be loaded in the dialogbox and will be ready for editing. Use the numbered Tabs on the upper left-hand side to show additional fields, if more than 15 data fields are available for a record.

Note that you can edit all fields, but your changes will only be applied to the MagIC SmartBook when you click the Close button, when you move through your records by using the Next and Previous buttons, or when clicking the New Record button. Temporary changes can be undone by Canceling the dialogbox.

New Record Button

To add a new record click the New Record button. With this action a new record is inserted at the bottom of the active table.

Delete Button

To delete the currently loaded data record from the MagIC SmartBook table click on the Delete button. You will be asked for a confirmation. Deletions cannot be undone.

Previous and Next Buttons

To move between adjacent data records click on the Previous or Next buttons. You can also make use of the Alt P and Alt N shortcut keys for faster navigation. Note that each data record preloaded in the Edit Data dialogbox is checked automatically, when you move to the next or previous data record, or when closing the dialogbox.

Prefill Records Checkbox

If you checked the Prefill Records checkbox, this new record will be pre-filled with the data from the currently-selected data record in the active table. Use this functionality when adding data records that have the majority of their data field entries in common. This might significantly improve the speed of data entry for larger data sets.

Tab Navigation and Hot Buttons

To facilitate the efficient editing of your data in the Edit Data dialogbox, you can make use of the colored Hot Buttons on the right-hand side of this dialogbox, which will open Pop-Up dialogboxes (see below for two examples). These Hot Buttons can be divided into 3 different categories:

Large Text Edit Box

The Purple Buttons will provide you with a larger edit box to enter Long Text Entries.

Lists and Controlled Vocabularies

The Light Green and Light Blue Buttons provide you with Shortlists to select a single predefined item or multiple items (see example below). In case of the Light Blue Buttons your selection will create links towards other data tables in the MagIC SmartBook file. To select multiple items simply click on as many records in these lists as required.

Note Each Shortlist is also listed on the http://earthref.org/MAGIC/shortlists.htm webpage. Here the user can review the current Controlled Vocabularies and add new items, if the current lists are in omission.

Dynamic Listbox Entries

Dark Blue Buttons help you enter names of Authors and Editors in the ER_citations table in the proper format, as expected for the MagIC and EarthRef.org databases. Use the add, edit and delete buttons to enter the correct author and editor information. These Dark Blue Buttons also give interactive dialogboxes to allow you to enter multiple keywords, ocean names, country names, etc.

Note When entering the Author or Editor names apply the following formatting rules. Start with the Last Name followed by the Initials, while separating these with a comma. The Last Name should always start with a capital letter and you can enter up to four different Initials per name as separated by periods. You can also provide Initials in the Th.H. or T.-H. format for uncommon name styles. If you enter the Author or Editor names incorrectly, an error message will be generated, preventing you to add the names.

3.5.4 MagIC Wizard

Depending on the type and characteristics of a study, certain fields in the MagIC SmartBook tables may not be relevant and should be hidden. The MagIC Wizard can be launched from the MagIC Toolbar (or Ctrl Shift W) and allows you to customize the SmartBook in three steps. In effect, this tool will temporarily hide some tables and columns from view in Microsoft Excel^© depending on the options you selected. Note that the selections made in the Wizard are only applied after clicking the Finish button.

In each of the three wizard steps (see next page an example) you can Check or Uncheck the checkboxes that are relevant to your study. As you will see, these options go from general in the first step to detailed in the following steps. While unchecking these checkboxes, the wizard may uncheck other checkboxes by default. You can always recheck these checkboxes again, if you require that data to appear in your MagIC SmartBook file.

Show All and Clear All Buttons

You can show all fields again by clicking the Show All button in the Wizard, followed by clicking the Finish button. Alternatively, you can click the Clear All button to uncheck all checkboxes.

Use Predefined Settings

You can also use Predefined Settings to more easily apply the needed settings. When you click the Use Predefined Settings ... button, another dialogbox appears with options like Classical Directional Study, Classical Intensity Study, Modern Paleomagnetic Study, Modern Stratigraphic or Drill Core Study, and so on.

3.5.5 Navigating a SmartBook

Sort

The data in each table may be sorted in an ascending order using up to three columns as keys. To sort on only one column, select any cell in this column and click on the Sort button. To select two or three columns for sorting, push the control key and click on any cell to select the first, second and third column, and then click on the Sort button. The order in which you selected the sorting columns also determines the order in the sorting keys. Note that this action cannot be undone.

Home Button

With this button you can move to the First Column in the active table without scrolling vertically. You can also use Ctrl Shift Home on your keyboard.

Left and Right Buttons

To move multiple columns use the left and right buttons on the toolbar, or hold down the Ctrl Shift keys while using the Arrow Keys on your keyboard. This will allow you to move from one set of columns that are visible in the active Microsoft Excel^© window to the next (or previous) set, showing you at least one or two columns overlap. These actions do not cause any vertical scrolling.

End Button

With this button you can move to the Last Column in the active table without scrolling vertically. You can also use Ctrl Shift End on your keyboard.

Tables Menu

Use this menu to navigate between different Tables in the active MagIC SmartBook file. This pull down menu will be automatically updated every time you switch between different SmartBooks using the SmartBooks toolbar menu (see below).

SmartBooks Menu

You can switch between different MagIC SmartBook files using the SmartBooks toolbar menu. If you activate another file through this menu the Tables menu (see above) will be automatically updated.

Note We recommend that you use this menu option instead of the Microsoft Excel^© Window menu. Note that if you switch between different MagIC SmartBook files using the Window menu, the Tables menu may (in some cases) not be updated automatically.

3.5.6 Text Tools

Add Text Tool ...

Select the cells to which you want to add some text or which ones you which to remove or overwrite with some other text, in batch mode. Select the Add Text Tool option from the Text Tools menu. In this tool first type the Text to add ... and then select the Action you want to perform on the right hand side. Apply these settings by clicking on the Start button. Note that this function cannot be undone.

Number Conversion Tool ...

Select the cells on which you want to perform some simple conversions. Then select the Number Conversion Tool option from the Text Tools menu. Type in the Conversion value ... and select one of the actions on the right hand side. Click on the Start button to start applying the conversions. You only can perform one operation at a time.

Remove All Spaces

Select the cells from which you want to remove all spaces. Select the Remove All Spaces option from the Text Tools menu. Note that this function cannot be undone.

Remove Extra Spaces Only

Select the cells from which you want to remove extra spaces only. Select the Remove Extra Spaces Only option from the Text Tools menu. This function will remove all Double Spaces and any existent Leading and Trailing Spaces. Note that this function cannot be undone.

Remove All Characters

Select the cells from which you want to remove all characters with the exception of numbers. Select the Remove All Characters option from the Text Tools menu. This function will remove all Non-Numeric Characters, except for the "-" (minus) and "." (period) symbols, so that it leaves only numeric values. It will also remove all Spaces. Note that this function cannot be undone.

Remove Parentheses and Brackets

Select the cells from which you want to remove all parentheses and brackets. Select the Remove Parentheses and Brackets option from the Text Tools menu. This function will remove all the "( )", "[ ]" and "{ }" symbols. Note that this function cannot be undone.

Remove Strange Symbols

Select the cells from which you want to remove strange symbols. Select the Remove Strange Symbols option from the Text Tools menu. This function will remove the "~!@#$%^&*_`'|<>? =" symbols and the "tab" character. Note that this function cannot be undone.

Remove Line Feeds

Select the cells from which you want to remove line feeds and carriage returns. Select the Remove Line Feeds option from the Text Tools menu. Note that this function cannot be undone.

Make All Lower Case

Select the cells for which you want the text to appear in all lower case. Select the Make All Lower Case option from the Text Tools menu. Note that this function cannot be undone.

Make All Upper Case

Select the cells for which you want the text to appear in all upper case. Select the Make All Upper Case option from the Text Tools menu. Note that this function cannot be undone.

Make Normal Case

Select the cells for which you want the text to appear in normal case, meaning that only the first letter of the text will appear in upper case, while the remainder will appear in lower case. Select the Make Normal Case option from the Text Tools menu. Note that this function cannot be undone.

Make Every Word Normal Case

Select the cells for which you want the text to appear with each word having normal case. This means that the first letter of each word in the text will appear in upper case, while the remainder of these words will appear in lower case. Select the Make Every Word Normal Case option from the Text Tools menu. Note that this function cannot be undone.

3.5.7 MagIC Help Menu

Metadata and Data Model

Select the Metadata and Data Model option from the MagIC Help menu to launch your browser and to review the current metadata and data definition. When entering this web page you will first see an overview of all Tables that define the MagIC SmartBook files. Click on a Table Name to view the definitions of the Records (columns) that define these tables, including short explanations and examples. Note that this function is equivalent to following the http://earthref.org/MAGIC/metadata.htm link. On this web form you can also do a free text search.

Note You can also review the Metadata and Data Definition by using the MagIC.metadata.definition.vXX.xls file that is part of the download of this MagIC Package. If you cannot locate this file on your local system, please follow http://earthref.org/MAGIC/software.htm and download this file separately from ERDA.

Method Codes

Select the Method Codes option from the MagIC Help menu to launch your browser and find a listing of Method Codes that currently are in use. On this page you will find a list of method types (or groups) for which we have defined codes. Please click on one of these method types to retrieve a list of relevant Method Codes and their definitions. Note that this is a dynamic list. With every new version, new Method Codes may have been added to the system, and will show up in these listings. This function is equivalent to following the http://earthref.org/MAGIC/methods.htm link. On this web form you can also do a free text search.

Instrument Codes

Select the Instrument Codes option from the MagIC Help menu and find a listing of Instrument Codes that currently are in use in the MagIC Database. Please click one of these codes to view their definitions. Note that this is a dynamic list. Every time a user adds new data, new Instrument Codes may have been added to the system, and will show up in these listings. This function is equivalent to following the http://earthref.org/MAGIC/instruments.htm link. On this web form you can also do a free text search.

Controlled Vocabularies

Select the Controlled Vocabularies option from the MagIC Help menu to find a listing of all Keywords that can be used when entering your data using the MagIC Console Software. On this page you will find a listing of Controlled Vocabularies for which we have defined keywords. Please click on one of the listed vocabularies to retrieve a list of (optional) keywords. This function is equivalent to following the http://earthref.org/MAGIC/shortlists.htm link. On this web form you can also do a free text search.

3.7 Known Bugs and Workarounds

This section contains a list of know Bugs and Workarounds to solve these problems. If you encounter new bugs, we would appreciate it if you could email us and provide us with information that describes the circumstances under which you encountered the problems. Thanks!

3.7.1 Object Library Not Registered (Macintosh OS X)

If you encounter an Object library not registered error message on Macintosh^© OS X systems, your Microsoft Office^© installation contains a corrupt library/preferences/Microsoft/Office Registration Cache X file. Open this file in a text editor, where you should remove all text and save it. Finish by locking this file and restarting the Microsoft Excel^© application. The locking of this text file is most essential, otherwise the problem will re-occur.

3.7.2 Appearance of Blank MagIC Toolbar (Macintosh OS X)

When starting the MagIC Console Software by double clicking on the MagIC.vXX.console.xls file in the Finder or from the Desktop, the software might start up displaying a blank MagIC toolbar. This problem can be circumvented by first launching the Microsoft Excel^© application itself, followed by opening the MagIC.vXX.console.xls file through the File # Open menu. In some cases, updating to Service Pack 1 will remedy this problem as well.

Contents	MagIC
Chapter 1	Introduction
Chapter 2	MagIC Website
Chapter 4	Importing Data With PmagPy and MagIC.py
Chapter 5	MagIC Glossary


			MagIC \| Help Contents \| Full Text
MagIC Help Library
			First \| Previous \| Next \| Last