Mnesia Table Fragmentation
http://erlang.open-source-related.info/erlang-formula/Erlang-Community-Mnesia-Table-Fragmentation-Trapexit-a-618.html
Contents |
[edit]Overview
This HOWTO describes how you make Fragmented Mnesia tables and how to use them.
[edit]Requirements
Let's say I have to make a book library index application. There's a table I use to record all the available library books. The record structure is as below. Due to the high volume of data, I want this table to be fragmented in a single Erlang node. If you want to make this fragmented table distributed, you may refer to http://www.trapexit.org/Distributing_a_Mnesia_schema tutorial on making distributed table. All the rest of the work related to table framentation remains same.
[edit]Sample Fragmented Table
I need this table to be disk_copies. Other modes also operate the same way.
-record(book_info, {isbn, name, author, keywords, category, description}).
[edit]Start an Erlang node
Our example node foo@example has the default disc storage path set to the directory Mnesia.foo@example in the current directory.
erl -sname foo
The directory can be overridden by using -mnesia dir '"/path/of/your/preference"' ' when starting the node.
Let create a disk based schema by running,
mnesia:create_schema([node()]).
[edit]Create the Fragmented table with 20 table fragments
In this example all 20 fragments are in the same Erlang/Mnesia node. Also the fragments are disc_copies.
mnesia:create_table(book_info,
[{frag_properties, [{node_pool, [node()]}, {n_fragments, 20}, {n_disc_copies, 1}]},
{index, [name, keywords, category]},
{attributes, record_info(fields, book_info)}]),
[edit]Data operations
In order to be able to access a record in a fragmented table, Mnesia must determine to which fragment the actual record belongs. This is done by the mnesia_frag module, which implements the mnesia_access callback behaviour. Wrap standard Mnesia operation functions inside the function and pass to mnesia:activity/4 with callback module mnesia_frag.
[edit]Add records
Create a function which calls mnesia:write/3.
AddFun = fun() ->
mnesia:write(book_info, Record, write)
end
Now call that function inside mnesia:activity/4 as below.
ok = mnesia:activity(transaction, AddFun, [], mnesia_frag)
Notice that I have used the activity access context as "transaction". Transaction makes sure that the operation is all successfull or all fail (atomic). AccessContext I can use are,
{transaction, Retries}
sync_transaction
{sync_transaction, Retries}
async_dirty
sync_dirty
ets
For example if you want to do above activity in dirty mode, you can write,
ok = mnesia:activity(async_dirty, AddFun, [], mnesia_frag) Refer to mnesia:activity/4 documentation for more info.
[edit]Select records with limit
As an example let's select books by Author "steve" with 10 books limit. Remember 10 is not a hard limit. Create a function with mnesia:select/4 function.
MatchHead = #book_info{author = "steve", _ = '_'},
Guard = [],
Result = ['$_'],
MatchSpec = [{MatchHead, Guard, Result}],
SelFun = fun() ->
mnesia:select(book_info, MatchSpec, 10, read)
end,
Now call that function inside mnesia:activity/4 as below.
Result = mnesia:activity(transaction, AddFun, [], mnesia_frag)
Result -> '$end_of_table' | {[Objects], Cont} | transaction abort
In a fragmented table, if it returns {[Objects], Cont} and the number of objects returned is less than the number of objects expected (10), you need to recursively run mnesia:select/1 with the return Cont (continuation) until you get the expected number of results or '$end_of_table'.
SelFun2 = fun() ->
mnesia:select(Cont)
end,
Result2 = mnesia:activity(transaction, SelFun2, [], mnesia_frag)
Result2 -> '$end_of_table' | {[Objects], Cont} | transaction abort
[edit] List of fragmented tables
To obtain the list of tables, use mnesia:table_info/2 with the option frag_names:
mnesia:activity(async_dirty, mnesia:table_info/2, [store, frag_names], mnesia_frag).
That's it! Now you know how to write a basic Mnesia fragmented tables program.
Copyright reserved by: www.trapexit.org