Using Metapack Packages¶
This section will use and example package, which is available online in three forms at:
Additionally, the section will refer to the filesystem package, which is just the ZIP package unpacked:
$ wget http://library.metatab.org/example.com-full-2017-us-1.zip
$ unzip example.com-full-2017-us-1.zip
Command Line Operations¶
Note
See the commands page for detailed documentation of the commands in the base metapack package.
Command line programs that operate on packages take a URL to a local or remote package, with a URL fragment to refer to a resource.
Metapack has a single CLI entrypoint, mp, to run commands. The entrypoint is exensible, so other python modules can install additinoal commands. You can see the commands that are avilable, and their version numebrs, with mp config
$ mp config
Package Version
------------- ---------
metapack 0.9.1
metatab 0.7.16
metatabdecl 1.6
rowgenerators 0.8.31
publicdata 0.3.8
tableintuit 0.1.5
Subcommand Package
------------ -------------------------------------
config metapack 0.9.1
doc metapack 0.9.1
index metapack 0.9.1
info metapack 0.9.1
open metapack 0.9.1
run metapack 0.9.1
search metapack 0.9.1
build metapack-build 0.0.2
edit metapack-build 0.0.2
new metapack-build 0.0.2
s3 metapack-build 0.0.2
stats metapack-build 0.0.2
update metapack-build 0.0.2
url metapack-build 0.0.2
wp metapack-wp 0.0.6.post0.dev1+g4c7a6bc
Commands which operate on packages take a URL as the last argument. There are a few common forms for these URLs:
Full url |
|
Local file |
example.com-full-2017-us-1.zip |
Metatab.csv in current directory |
. or ‘’ |
Commands that operate on resources use URLs that have a fragment and the name of the resource:
Full url |
http://library.metatab.org/example.com-full-2017-us-1.zip#renter_cost |
Local file |
example.com-full-2017-us-1.zip#renter_cost |
Metatab.csv in current directory |
.#renter_cost |
For instance, to list the resources in a package:
# Remote Package
$ mp info http://library.metatab.org/example.com-full-2017-us-1.zip
# Local filesystem package
$ cd example.com-full-2017-us-1
$ mp info -r
To display the resources:
# Remote Package
$ mp info http://library.metatab.org/example.com-full-2017-us-1.zip#renter_cost
# Local filesystem package
$ cd example.com-full-2017-us-1
$ mp info -r .#renter_cost
Note that in a filesystem package, the reference to the package (‘.’) can be omitted when referencing the package, but is included when referencing a resource. This is because the shell considers strings that start with ‘#’ to be comments. To avoid this, the ‘#’ can also be escaped, mp info -r \#renter_cost, but this form is not commonly used.
Opening Packages¶
Opening packages with the CLI primarily means opening the CSV files included in the package, which you can do from direct reference. Just open the metadata file and read the resource URLs. However, you can also use the mp run program to dump the data in a variety of formats. The URL requires a resource fragment, but if you don’t include it, mp run will show the resources.
$ mp run example.com-full-2017-us-1.zip
Select a resource to run:
Type Ref Url
--------- ------------------------ --------------------------------------
Resource #renter_cost data/renter_cost.csv
Resource #simple-example-altnames data/simple-example-altnames.csv
Resource #simple-example data/simple-example.csv
...
$ mp run example.com-full-2017-us-1.zip#renter_cost
id,col2,renter_cost_gt_30,renter_cost_gt_30_cv, ...
1,,1447,13.6176070905,42.2481751825,8.272140707
2,,5581,6.235932071,49.2803532009,4.9333693054
3,,525,17.6481586483,45.2196382429,13.2887199931
If you just want to list all of the resources and references, use mp info -r.
Using Packages In Python¶
Using packages in Python involves opening the package, and then acessing a resource.
import metapack as mp
# Open the package.
pkg = mp.open_package('http://library.metatab.org/example.com-full-2017-us-1.csv')
print(pkg) # display documentation
# Get a handle on a resource
r = pkg.resource('renter_cost')
# Get a dataframe for the resource
df = r.dataframe()