persistence/persistent_stl.hpp
Data Persistence for the STL Classes

Introduction

The template classes provided by the STL have been made persistent using template dump/restore functions. To avoid overload problems the functions are actually called dump_class and restore_class where class is the name of the template class. For example, the persistence functions for the STL map are called dump_map and restore_map.

Also provided here are the persistence functions for the std::string type, which are not templatised since std::string isn't. The functions for the various classes are explained in the folowing sections.

The persistence functions for templates are themselves templates, so are automatically adapted to the type that the container holds. For example, stlplus::dump_vector which is the dump routine for the STL vector, will adapt to the type being held in the vector. If the vector contains int, then the dump_vector function will dump ints by calling the dump function defined for int. If the vector contains std::strings then the dump_vector function will dump strings. The template function does this by taking the name of the element dump function as a parameter.

For example:

#include <string>
#include <vector>
#include "persistent_contexts.hpp"
#include "persistent_vector.hpp"
#include "persistent_string.hpp"

void dump_string_vector(stlplus::dump_context& context, const std::vector<std::string>& data)
{
  stlplus::dump_vector(context, data, stlplus::dump_string);
}

void restore_string_vector(stlplus::restore_context& context, std::vector<std::string>& data)
{
  stlplus::restore_vector(context, data, stlplus::restore_string);
}

The last parameter passed to the dump_vector and restore_vector functions is the name of the function to use to dump/restore the element type - in this case std::string.

The same parameter profile is used for all dump/restore routines, except that for classes that contain more than one type, there will be more than one function name required. For example, the std::pair contains two different types, so requires two function names for each call:

#include <string>
#include <map>
#include "persistent_contexts.hpp"
#include "persistent_pair.hpp"
#include "persistent_int.hpp"
#include "persistent_string.hpp"

typedef std::pair<std::string,int> string_int_pair;

void dump_string_int_pair(stlplus::dump_context& context, const string_int_pair& data)
{
  stlplus::dump_pair(context, data, stlplus::dump_string, stlplus::dump_int);
}

void restore_string_int_pair(stlplus::restore_context& context, string_int_pair& data)
{
  stlplus::restore_pair(context, data, stlplus::restore_string, stlplus::restore_int);
}

In this example, dump_string and dump_int manage the dumping of the two types, whilst restore_string and restore_int manage the restoration.

In both of these examples, the data structure can be dumped using only functions already provided by the persistence library. However, I prefer to wrap the function calls into a top-level function that has a simple interface and hides the implementation, as in the example above, even though the implementation is one line of code.

Note:The function name passed as a parameter must have only two parameters - the context and the data. To pass a function that takes extra parameters, wrap those functions in a wrapper function, like the dump_string_int_pair and restore_string_int_pair functions above. These functions can be used in a higher-level data structure, whereas the stlplus::dump_pair and stlplus:restore_pair cannot because they take more than two parameters.

Header persistent_stl.hpp

You can include all of the functions in one go by including persistent_stl.hpp. If you prefer however, you can include the separate headers for each class being used. The latter approach minimises the number of STL headers that get included indirectly.

persistent_string.hpp: Persistence of std::string

This set of functions provides the following interface:

// basic_string

template<typename charT, typename traits, typename allocator, typename D>
void stlplus::dump_basic_string(stlplus::dump_context&, const std::basic_string<charT,traits,allocator>& data, D dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename charT, typename traits, typename allocator, typename R>
void stlplus::restore_basic_string(stlplus::restore_context&, std::basic_string<charT,traits,allocator>& data, R restore_fn)
  throw(stlplus::persistent_restore_failed);

// string

void stlplus::dump_string(stlplus::dump_context&, const std::string& data)
  throw(stlplus::persistent_dump_failed);

void stlplus::restore_string(stlplus::restore_context&, std::string& data) 
  throw(stlplus::persistent_restore_failed);

The first pair of functions are template persistence functions for the templatised class std::basic_string which can be used to represent strings of any character width.

The second pair of functions are non-templatised functions for the persistence of std::string - the specialisation of std::basic_string for 8-bit characters.

Note:The 'standard' type std::wstring is not supported because it is too weakly defined. It is ostensibly the Unicode string, which requires a minimum of 21 bits. On Windows wstring is only 16 bits wide so is actually UTF-16 encoded Unicode. On Gnu/Linux, wstring is 32 bits wide and therefore is unencoded UCS-4. Since UTF-16 and UCS-4 are not compatible and are not easily convertible, it is not possible to implement platform-independent persistence of this type. I recommend that you convert wide strings into UTF-8 encoding and dump/restore them using the std::string persistence above.

Persistence of std::pair

This header provides persistence functions for the std::pair collection. This is defined in the <map> header but is useful enough to be regarded as a separate class in its own right.

The header provides the following functions:

template<typename V1, typename V2, typename D1, typename D2>
void stlplus::dump_pair(stlplus::dump_context&, const std::pair<V1,V2>& data, D1 dump_fn1, D2 dump_fn2)
  throw(stlplus::persistent_dump_failed);

template<typename V1, typename V2, typename R1, typename R2>
void stlplus::restore_pair(stlplus::restore_context&, std::pair<V1,V2>& data, R1 restore_fn1, R2 restore_fn2)
  throw(stlplus::persistent_restore_failed);

These functions take two parameters that are names of functions: one for each element in the pair. For example, if you were using a pair containing a string and an int:

#include <string>
#include <map>
#include "persistent_contexts.hpp"
#include "persistent_pair.hpp"
#include "persistent_string.hpp"
#include "persistent_int.hpp"

typedef std::pair<std::string,int> string_int_pair;

void dump_string_int_pair(stlplus::dump_context& context, const string_int_pair& data)
{
  stlplus::dump_pair(context, data, stlplus::dump_string, stlplus::dump_int);
}

void restore_string_int_pair(stlplus::restore_context& context, string_int_pair& data)
{
  stlplus::restore_pair(context, data, stlplus::restore_string, stlplus::restore_int);
}

Persistence of std::vector

The vector contains one element type and therefore is parametrised by one element function.

The interface is:

template<typename T, typename D>
void stlplus::dump_vector(stlplus::dump_context&, const std::vector<T>& data, D dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename T, typename R>
void stlplus::restore_vector(stlplus::restore_context&, std::vector<T>& data, R restore_fn)
  throw(stlplus::persistent_restore_failed);

// specialism for vector<bool>

void stlplus::dump_vector_bool(stlplus::dump_context&, const std::vector<bool>& data)
  throw(stlplus::persistent_dump_failed);

void stlplus::restore_vector_bool(stlplus::restore_context&, std::vector<bool>& data)
  throw(stlplus::persistent_restore_failed);

The first pair of functions are templatised functions for the persistence of a vector containing an element of any class and takes as its third parameter the function for managing the elements. For example, for a vector of string, the element functions would be stlplus::dump_string and stlplus::restore_string.

The second pair of functions are non-templatised functions for the persistence of a vector of bool, which has a specialised implementation in the STL and a slightly different interface.

Persistence of std::list

The list contains one element type and therefore is parametrised by one element function.

The interface is:

template<typename T, typename D>
void stlplus::dump_list(stlplus::dump_context&, const std::list<T>& data, D dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename T, typename R>
void stlplus::restore_list(stlplus::restore_context&, std::list<T>& data, R restore_fn)
  throw(stlplus::persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a list containing an element of any class and takes as its third parameter the function for managing the elements. For example, for a list of string, the element functions would be stlplus::dump_string and stlplus::restore_string.

Persistence of std::deque

The deque contains one element type and therefore is parametrised by one element function.

The interface is:

template<typename T, typename D>
void stlplus::dump_deque(stlplus::dump_context&, const std::deque<T>& data, D dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename T, typename R>
void stlplus::restore_deque(stlplus::restore_context&, std::deque<T>& data, R restore_fn)
  throw(stlplus::persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a deque containing an element of any class and takes as its third parameter the function for managing the elements. For example, for a deque of string, the element functions would be stlplus::dump_string and stlplus::restore_string.

Persistence of std::set

The set contains one element type and therefore is parametrised by one element function.

The interface is:

template<typename K, typename P, typename D>
void stlplus::dump_set(stlplus::dump_context&, const std::set<K,P>& data, D dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename K, typename P, typename R>
void stlplus::restore_set(stlplus::restore_context&, std::set<K,P>& data, R restore_fn)
  throw(stlplus::persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a set containing an element of any class and takes as its third parameter the function for managing the elements. For example, for a set of string, the element functions would be stlplus::dump_string and stlplus::restore_string.

Persistence of std::multiset

The multiset contains one element type and therefore is parametrised by one element function.

The interface is:

template<typename K, typename P, typename D>
void stlplus::dump_multiset(stlplus::dump_context&, const std::multiset<K,P>& data, D dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename K, typename P, typename R>
void stlplus::restore_multiset(stlplus::restore_context&, std::multiset<K,P>& data, R restore_fn)
  throw(stlplus::persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a multiset containing an element of any class and takes as its third parameter the function for managing the elements. For example, for a multiset of string, the element functions would be stlplus::dump_string and stlplus::restore_string.

Persistence of std::map

The map contains two element types and therefore is parametrised by two element functions.

The interface is:

template<typename K, typename T, typename P, typename DK, typename DT>
void stlplus::dump_map(stlplus::dump_context&, const std::map<K,T,P>& data, DK key_dump_fn, DT val_dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename K, typename T, typename P, typename RK, typename RT>
void stlplus::restore_map(stlplus::restore_context&, std::map<K,T,P>& data, RK key_restore_fn, RT val_restore_fn)
  throw(stlplus::persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a map containing an element of any class and takes as its third and fourth parameters the functions for managing the key and value respectively. For example, for a map of ints mapped onto string, the element functions would be stlplus::dump_int/stlplus::dump_string and stlplus::restore_int/stlplus::restore_string.

Persistence of std::multimap

The multimap contains two element types and therefore is parametrised by two element functions.

The interface is:

template<typename K, typename T, typename P, typename DK, typename DT>
void stlplus::dump_multimap(stlplus::dump_context&, const std::multimap<K,T,P>& data, DK key_dump_fn, DT val_dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename K, typename T, typename P, typename RK, typename RT>
void stlplus::restore_multimap(stlplus::restore_context&, std::multimap<K,T,P>& data, RK key_restore_fn, RT val_restore_fn)
  throw(stlplus::persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a multimap containing an element of any class and takes as its third and fourth parameters the functions for managing the key and value respectively. For example, for a multimap of ints mapped onto string, the element functions would be stlplus::dump_int/stlplus::dump_string and stlplus::restore_int/stlplus::restore_string.

Persistence of std::complex

A std::complex is like a pair that contains two elements, except that they must be of the same type. For example, a complex<int> or complex<float>.

template<typename T, typename D>
void stlplus::dump_complex(stlplus::dump_context&, const std::complex<T>& data, D dump_fn)
  throw(stlplus::persistent_dump_failed);

template<typename T, typename R>
void stlplus::restore_complex(stlplus::restore_context&, std::complex<T>& data, R restore_fn)
  throw(stlplus::persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a complex containing an element of any class and takes as its third parameter the function for managing the contained type. For example, for a complex of int, the element functions would be stlplus::dump_int and stlplus::restore_int.

Persistence of std::bitset

A bitset is an array of bits where the size is set at compile time, unlike vector<bool> where the size is dynamic. A bitset is unusual in that it is templatised with the size as the template parameter. This means that bitsets of different sizes are in fact different types and cannot interact.

template<size_t N>
void dump_bitset(dump_context&, const std::bitset<N>& data)
  throw(stlplus::persistent_dump_failed);

template<size_t N>
void restore_bitset(restore_context&, std::bitset<N>& data)
  throw(stlplus::persistent_restore_failed);

These functions take two arguments, so can be used as parameters to higher-level persistence functions. For example:

#include <bitset>
#include <vector>
#include "persistent_contexts.hpp"
#include "persistent_bitset.hpp"
#include "persistent_vector.hpp"

typedef std::bitset<100> flags;
typedef std::vector<flags> flags_vector;

void dump_flags_vector(stlplus::dump_context& context, const flags_vector& data)
{
  stlplus::dump_vector(context, data, stlplus::dump_bitset<100>);
}

void restore_flags_vector(stlplus::restore_context& context, flags_vector& data)
{
  stlplus::restore_vector(context, data, stlplus::restore_bitset<100>);
}

Persistence of std::shared_ptr

The shared_ptr class is part of the "C++11" extensions to C++. This standard has now been ratified, and some compilers support some of its features. For this reason, support for persistence of shared_ptr has been added to the STLplus persistence library, but only for those compilers that support it.

Note: the shared_ptr class was pre-released as part of the "TR1" proposed extensions which eventually became part of the "C++11" standard. I do not support shared_ptr in its TR1 form because it's implementation has varied between compilers too much to provide a portable solution. The feature must be used in its final, standard-compliant form.

At the time of writing the following compilers support shared_ptr:

Microsoft Visual Studio
From Visual Studio Express 2010 the C++11 form of shared_ptr is provided by default.
Gnu gcc
The C++11 form of shared_ptr is supported in gcc from version 4, provided you enable the c++11 mode with the command-line switch -std=c++11. It is available in TR1 form without the switch but I do not support that version.
Borland (now Embarcadero) C++ Builder
The Borland compiler does not provide shared_ptr as yet (checked in v6.31). It uses the Dinkumware STL library as does Visual Studio, but an older version.

The shared_ptr class contains a pointer to one element type and therefore is parametrised by one element function.

The interface is:

template<typename T, typename DE>
void dump_shared_ptr(dump_context&, const std::shared_ptr<T>& data, DE dump_element)
  throw(persistent_dump_failed);

template<typename T, typename RE>
void restore_shared_ptr(restore_context&, std::shared_ptr<T>& data, RE restore_element)
  throw(persistent_restore_failed);

This pair of functions are templatised functions for the persistence of a shared_ptr containing an element of the template class and takes as its third parameter the function for managing the elements. For example, for a pointer to a string, the element functions would be stlplus::dump_string and stlplus::restore_string.

Note: the functions for implementing pointers to polymorphic classes haven't been completed yet. See the discussion for implementing persistence with polymorphism for the STLplus smart pointers for more information.

Omitted Classes - Adaptors and Iterators

Note: I do not support persistence of the container adaptors queue, priority_queue and stack because their interfaces are too restrictive to allow dump and restore routines to be written without burgling the data structure. This means that I will never do them because it is impossible!

When designing a data structure to be made persistent, you need to bear this in mind and use containers such as vector and list rather than queue or stack.

I also haven't implemented any STL iterators. The design of iterators makes it nearly impossible to do this without burgling the data structure, which wouldn't be portable.