This section is an attempt to document all the changes made to the STLplus library. However, the change log has been restarted for version 3 of the library since the changes for this version are huge. Therefore only the changes since then have been documented.
This was a legacy part of the message handler which originally got its default language messages from this file. For some time now it has used built-in strings for the default values so only locale-specific translations use message files, so it is time this file was deleted.
Inheriting from std::iterator is deprecated in C++17.
In order to make the STLplus containers conformant, I have inlined the typedefs required to specify iterator traits and removed the inheritance.
I was curious about why the standards committee would do this, so I did a bit more research.
The rationale for this change is here:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0174r1.html#2.1
It seems to make sense, the committee have recognised that typedefs are not inherited quite how you expect them to be, so using inheritance of std::iterator to define iterator traits doesn't work. The STL has worked round this for years. Presumably the problem is shown when using iterators in algorithms that use traits for optimisation, I have never tried this with STLplus iterators. So the old recommendation of inheriting std::iterator, as stated here:
http://www.cplusplus.com/reference/iterator/iterator/
Which says "This is a base class template that can be used to derive iterator classes from it." has been reversed and now inheriting from std::iterator is deprecated and the recommendation is to define traits yourself, inlined in the iterator declaration.
It turns out that this is a trivially easy change to make, so I have inlined the traits for all iterators in the containers library.
Experiments on FreeBSD (the TrueOS distribution) confirm that STLplus builds with clang instead of Gnu gcc. Small changes, such as the reintroduction of a small number of exception specifications mentioned below were necessary, but no functional changes were needed. The portability/build.cpp utilities now recognises a clang build and correctly identifies the compiler and version.
STLplus has exception specifications, e.g. throw(exception1,exception2), after function declarations so that both the compiler and user know what exceptions the function can throw.
http://en.cppreference.com/w/cpp/language/except_spec
However, in C++11 the use of throw specifications is deprecated and in C++17 they have been removed. The rationale for this change is explained here:
http://www.gotw.ca/publications/mill22.htm
This started to show up as compiler warnings in gcc v7.2 and in Visual Studio. I have removed all of the throw specifications from STLplus. Where possible in the headers I have turned them into a comment so there is still some user information about what exceptions might be thrown by a function.
I have left empty throw() specifications on exception classes derived from system exceptions because otherwise I was seeing compiler errors due to the derivative having a different throw specification than its base class. This specific empty form of exception specification is being kept for the time being to manage the transition to C++17 and is equivalent to the directive noexcept(true).
http://en.cppreference.com/w/cpp/language/noexcept_spec
Traditionally, the STLplus code has been littered with macros to allow for differences in support for C++ features between compilers. This was always difficult to maintain. The fixes are localised in each library to a single header called <library>_fixes.hpp.
I have decided to scrap support for old compilers that require these work-arounds. These are:
This decision has enabled significant simplifications to the code and now nearly all the fixes are gone from the code - it is almost exactly the same on all platforms and all compilers. It is a relief at last to see C++ standardisation reaching this point.
Patch submitted by DJDM:
Added hash::at() and hash::at_pointer() methods to get the value associated with a key directly, avoiding the use of iterators. The motivation was to avoid a problem with iterators which are not thread-safe, but these are also generally useful functions.
The hash::at() method has the same functionality as the hash::operator[] method in read-only mode, but does not have the catch that in read-write mode, [] creates a new value if the key is not found. The at() method throws an exception instead when the key is not found. The at_pointer() method returns a pointer to the value, rather than a reference and returns null if the key is not found.
This component was written originally to avoid the buffer-overrun problem of sprintf, it formats text in the same way as sprintf but into a std::string.
This was written originally to unpick the format string and use sprintf in a controlled way that ensured buffer overrun could not happen. It was rewritten a few years ago to use platform-specific functions that did their own low-level memory allocation, eliminating the need to unpick the format string. This solution was non-standard but used commonly available functions. However, one of these common functions (vasprintf) has been removed from some versions of the C runtime (specifically Cygwin) so I have written a more standard-conformant version using vsnprintf to print to a buffer.
Note: there are local copies of these functions in strings/string_float and strings/string_utilities which were added to eliminate a dependency from strings library to portability library. These have been changed to the new implementation as well.
Bug reported by Simone D: The old digraph::path_exists algorithm was incorrect - it would stop searching for a path if an arc pointed to an already-visited node but before visiting any sibling arcs which might have been candidate paths.
I have rewritten the path_exists algorithm using a simpler method.
This is a minor bug-fix so that paths are simplified more consistently. For example "/.." can be simplified to "/".
Note: The simplification of paths used in file_system is a naive algorithm, it removes ".." elements on the assumption that none of the path is made of links, so that "/root/element/.." always maps onto "/root". This is not necessarily true.
Some of the IDE project files still contained references to source files strings/print_address.hpp/cpp and strings/string_address.hpp/cpp which are obsolete - these functions were merged into other source files at the last release. These references have been removed.
These are a simple conversion of the VS 2012 project files, but have their own IDE folder so that they can be kept separate for users running more than one version of Visual Studio
Patch submitted by DJDM - extend the string formatting functions to support long long types.
On OS-X the option for preventing a subprocess read from triggering a signal has a different name from other Unix-like OSs. Fix supplied by Ingwie.
The ntree class is an ordered tree, but feedback has indicated that it is lacking in some methods for interchanging integer offsets of children with their iterators and manipulating the order of the tree once it has been built. These operations are possible, but difficult and should be built-in.
So, the following methods have been added:
See the documentation for ntree to see how these are used.
There is now a method called move in the digraph class which allows a whole graph to be moved into another graph, merging the two graphs. All arc connectivity is preserved in the move and all iterators to any nodes or arcs in the moved graph will continue to be valid and work.
You can now test whether an iterator is owned by a graph and therefore can be used with it by using the owns method. The method declaration is:
bool digraph::owns(iterator);
Note that a null iterator is not owned by any object. Valid or end iterators are owned and may only be used with their owner object.
The option of erasing all the children of a node but keeping the node was missing from the ntree class, so I've added a method called erase_children.
Previous versions allowed a single child to be deleted with the erase(i,offset) method. I've given this a synonym of erase_child for consistency with the new erase_children method. You can use either according to your preference.
Implemented a breadth-first traversal algorithm as a method of the ntree class. This method returns a vector of iterators in traversal order. The traversal algorithm is really too complex to implement as a new kind of iterator.
This approach is consistent with the digraph class which implements complex algorithms as methods returning vectors of iterators as results.
A bug got introduced caused by a mismatch between the codes for the Posix codes EINPROGRESS etc. and the Windows sockets variants WSAEINPROGRESS etc. I had assumed that Microsoft had mapped one set onto the other (previous versions of Visual Studio didn't define the Posix codes at all).
The bug only shows itself with the Visual Studio compiler - so for example using MingW to compile a native Windows program works.
Unfortunately it also shows up a bug in the test pack (actually, two) so the problem didn't get caught in testing. This is why the bug made it out into the field.
The STLplus iterators gave a compiler error when used with standard algorithms such as std::find. I have made the iterators inherit from std::iterator to fix this.
STLplus has supported 64-bit builds for a long time, but gave a number of compiler warnings. I have eliminated these warnings so the library collection should now compile cleanly in both 32-bit and 64-bit modes.
I have added the 64-bit target as an option in the Visual Studio 2012 project files. So STLplus can now be compiled in either Win32 mode or x64 mode.
There are now project files for Visual Studio 2012 as well as 2010. Pick the correct one for the version of the compiler you are using.
Developers: there is now an IDE directory in the tests directory which contains separate project files for the different Visual Studio versions.
There were inconsistencies in the Visual Studio 2010 project files which showed up when converting them to 2012 project files. So I have regenerated the project files from scratch.
It is technically incorrect to label GNU/Linux as Linux. Linux is the Kernel, whereas GNU is the OS and both deserve credit. So I have changed the documentation, code and build tools to use the correct OS name of GNU/Linux.
This also means that the build subdirectory is now e.g. GNULINUX-i686-debug rather than LINUX-i686-debug.
The gNewSense distribution of GNU/Linux is a pure free software distribution. I have tested STLplus on this platform and confirmed that it works correctly. The only limitation is that you must install the development headers for the "ltdl" library, i.e. package "libltdl-dev" to get the dynalinker component to build.
The basic find algorithm was inefficient - so DJDM implemented a faster find method for internal use only.
Some of the containers referenced internal methods without the
this-> prefix. This is now required by the latest version of g++,
so I have added the required prefix in all cases where the compiler
raised an error.
A user (dark_knight_ita) reported a bug in digraph whereby removing the first or last node or arc left the internal pointers pointing at deleted memory.
This has been corrected by a patch also provided by the user.
DJDM reported a situation in which a simple_ptr had been double-destructed, causing a crash. OK, double-destruction should never happen anyway, but it is also simple to make these pointer classes proof against this as well.
DJDM contributed the double-destruct proofing of simple_ptr and I (AJR) adapted this for smart_ptr. This code only deals with avoiding double-destruction issues, it does not allow method-based access of a destroyed object.
The ntree::cut method had a bug (reported by sacbor) where the old method name set_new_owner was used when the renamed method change_owner should have been used. This bug was fixed in v3.6 but seems to have crept back in, possibly in the rework of ntree which was done for v3.7. So I've fixed it again.
Contributed by DJDM.
GCC is a little more rigid with it's types when it comes to using derivative template types being returned from template functions of a template. It is also a bit more rigid when it comes to accessing the private members of one instance of a template from another instance.
The project files for various IDEs are multiplying. So to manage these files better, I have created a separate top-level directory to contain them. There is a separate sub-directory for each supported IDE.
At present the following IDEs are supported:
There is a project file for building STLplus as 5 separate libraries and another for building the single monolithic library.
The gcc make system has not been changed and the Makefiles are still local to each folder in the build. This works better in my opinion for command-line make.
There is now support in the persistence library and in the strings library for the std::shared_ptr class. This is only supported if it is a true c++0x version of shared_ptr - that is, it is in the std:: namespace.
I don't support std::tr1::shared_ptr nor boost::shared_ptr.
In Visual Studio 2010, std::shared_ptr is available as standard.
In gcc version 4.5 (I'm not sure the earliest version), it is enabled by the gcc option -std=c++0x or by running make as:
make CXXFLAGS=-std=c++0x
In Borland (Embarcadero) C++ Builder, std::shared_ptr is not available.
In order to support this, I've made internal changes to the persistence context classes and to the implementation of persistence for smart_ptr, simple_ptr, digraph and ntree. However, these have no impact on user code and there is no change in the persistence file format.
Added downloadable code for some of the persistence examples.
A couple of bugs fixed related to the conversion from STLplus 2. These only showed when trying to compile a program that uses these functions.
Visual Studio now defines constants such as EINPROGRESS so I no longer need to define these. However, I still define them for older versions of Visual Studio.
The functions for differentiating between a file and a folder did not handle 'other' filesystem objects correctly. I have now defined a file to be a regular file (any OS), symbolic link (Unix only), socket (Unix only) or named pipe (any OS). A folder is defined as just a directory (any OS). Character devices (any OS) and block devices (Unix only) are neither files nor folders.
Two makefiles source/Makefile and tests/Makefile, relied on the current directory ('.') being on the path, which for me it always is. However, this caused make to fail for other users. The makefiles have been fixed so they do not rely on this.
Visual Studio 10 Express Edition is now the preferred offering from Microsoft. Converting the solution and project files from Visual Studio 9 overwrote them, so now there are only Visual Studio 10 solutions and projects.
In principle I could rename these to support both simultaneously, but I will wait and see if there is a demand before investing the time and effort to do so.
These fields were private so that derived classes couldn't access them. This was orignally chosen to hide the platform-specific nature of the internals. However, there are occasions when a subclass needs to access these fields, so they are now protected.
Change requested/contributed by DJDM.
On some platforms the external variable environ has to be declared before use. The declaration was accidentally moved inside the stlplus namespace. This doesn't show as a problem on other platforms because they supply system declarations of this variable. However, on NetBSD the system declaration is missing and this shows up the scoping problem.
The problem has been solved by simply moving the external declaration back into global scope where it belongs.
Bug fix contributed by DJDM
On Gnu/Linux, occasionally, if a socket disconnects remotely between the points at which you ask for send_ready and actually doing the send, then the send will fail, but also on Gnu/Linux sends the SIGPIPE signal, which kills the program if it is unhandled - most of the time. As the return of -1 is all you need from the send function, it's best to set it not to generate the SIGPIPE in the first place, which is done by adding the MSG_NOSIGNAL flag to the send calls.
You can use the portability/build functions to report which version of compiler and build options were used to build STLplus. I've tidied this up and provided more functions to get parts of the build string.
The getcwd function returns the C-string path to the current directory. This is then used to construct a std::string. However, getcwd can very rarely return a null pointer, which crashes the std::string constructor. I've fixed the code so that a null return value is converted into an empty string.
Some of the error handling was inconsistently handled in the subprocess classes, so I've worked through systematically adding error handling in a consistent way.
I've also added get_variables methods to the subprocess classes so you can examine the environment of the subprocess.
Fix contributed by DJDM
Having operator< allows these classes to be used in maps and sets. They are defined in terms of the operator< only for each of the contained types.
The ntree container was lacking a set of methods for building trees out of other trees without copying the contents. This is inefficient. I have added a set of move methods which move the argument tree into the target tree, destroying the argument tree in the process. These complement the insert methods that add subtrees in the same way, but copy them first.
In the process, I've deprecated the append method in favour of an insert or move with no offset, which defaults to the append functionality.
You can now merge the libraries together into one large library - known as a monolithic build of
STLplus. The simplest way of doing this is to use a shell - DOS shell on Windows, bash shell on
Gnu/Linux or other Unix - and cd to the stlplus3/source directory. There, run the script
make_monolithic. This copies all of the source code into the directory. Then you will
find there are alternative project files and build scripts in the source directory to allow you to
use STLplus as a monolithic library.
The simple_ptr variants of the smart_ptr classes were originally in v2 of STLplus but were not added to v3 because I was unsure whether to have two different ways of doing the same thing (something I do not generally agree with). However, the author (Dan Milton) has convinced me that they should be in the library because they are sufficiently different from smart_ptr.
The simple_ptr classes have been rewritten in the same style as smart_ptr and share the same set of copy functors to implement the copy/clone/nocopy semantics. To achieve this, the copy functors have been factored out into the header copy_functors.hpp.
Support functions for simple_ptr classes have been added to the persistence and strings libraries.
It has been reported that some of the pragmas in, for example containers_fixes.hpp which disable warnings in Borland compilers are now unnecessary. I have reduced this to a set of two:
These fit into the category of warnings you cannot get rid of which erroneously suggest there is a problem in STLplus when the problem is in the compiler. I dislike such warnings which raise false alarms, so they will remain disabled.
This method, which allows a hash element to be erased via its iterator, has been added for compatibility with std::map.
This function was present in STLplus v2, but was removed temporarily due to the removal of TextIO from STLplus. The intention was to re-add it soon after by porting the function to use IOStreams, but it got forgotten in the changeover. It has now been re-added, ported to IOStreams.
There was a bug reported in smart_ptr whereby a gcc-compiled application would crash due to a missing assignment operator in the baseclass smart_ptr_base. This bug was introduced in smart_ptr's redesign for STLplus v3.0 when the common base class was introduced.
Previous versions of STLplus had an object-oriented interface to TCP sockets, that is sockets which have been configured for TCP communications. This subsystem (part of portability) has been redesigned to provide three subsystems:
The ntree is a tree data structure where each node can have any number of children. This is technically not an n-ary tree as I thought when I wrote it. An n-ary tree has exactly n children on each node, whereas my ntree class can have any number of children. Furthermore, trees can be classed as rooted or unrooted depending on whether it has a single root node. So the documentation now refers to it as a rooted tree. That is, it is an arbitrary-branched tree with a single root node.
The code has not changed - in particular the class is still called ntree - because I don't see the point in breaking existing code over a slight terminology mix-up.
As has happened several times in the past, a new version of gcc has tighter rules than older versions and caused some minor problems in the code to be revealed. There are two types: (1) use of C++ features that were common practice when STLplus was originally developed, but which have been deprecated; (2) missing system includes, since some system files used to be implicitly included.
Both STLplus v3.4 and all the tests now compile correctly with gcc v4.3 as well as older versions.
The old STLplus v2 was ported to Borland v5.5 by a user. However, STLplus v3 has never been ported.
The libraries have been modified to support both the v5.5 compiler (the Free Command-line Tools) and the v5.8 compiler (Turbo C++ 2006 Explorer).
This has required the following changes:
The purpose of this release is to change the status from beta to stable. There are no code changes and the only difference is a correction in the copyright notice.
The copyright statement now gives credit to Southampton University, UK, who initially funded my work on this project and therefore have part-ownership of the STLplus project. The copyright statement now reads:
Copyright: (c) Southampton University 1999-2004
(c) Andy Rushton 2004-2008
This has no effect on licensing.
This class is a platform-independent and object-oriented interface to the dynamic library loader functions.
I have modified the persistence and strings libraries so that the dependency on the portability library is soft not hard. This means that both libraries can be compiled without the portability library if you don't want persistence or printing support for the portability/inf type.
The containers library had only one component that needed compiling - the rest of the library is template code that gets included in your code. I have inlined the one component (the exception classes) so that the library is entirely headers and does not need compiling.
This change also required changes to the makefiles project, so if you are using my makefiles you will need to download the latest version of that too (i.e. at least v1.4).
The STLplus library does not support Unicode (for good reasons - the wide string class is too weakly defined to have a common cross-platform interpretation). However, it should be usable in a system that does. I have modified the Windows system calls to always call the ANSI variants and use narrow strings even when compiled with the UNICODE macro enabled.
The function file_set_mode has been removed because it is non-portable and there's no way to make it portable. It was legacy code from a Unix-only project and didn't work on Windows. For the time being, use the platform specific functions for this. In the future I may re-introduce a limited functionality interface to the file modes once I can work out the common subset.
The macro END_CLI_DEFINITIONS contained an error with namespaces - so it would only work if you were "using namespace stlplus". This should now work when using the stlplus:: prefix notation as well.
There was a bug related to the use of a definition_t to define a command-line argument. The bug caused the cli_parser to crash with a segmentation fault due to a null pointer being passed to a std::string.
The minimum set of messages (for the cli_handler) has now been built-in to the message_handler so that applications can more easily be built with no message-file handling.
This bug resulted in a quoted string being converted by the argument_vector class into a doubled-up string - i.e. "./fred" would get converted into "..//ffrreedd". The bug only manifested on Windows.
A bug fix was submitted by Alistair Low to enable the subprocess classes to kill a whole subprocess tree rather than just the parent process. I have yet to confirm whether this is the behaviour of the Unix port - it should be.
One of the macros in the debug header had a typo - which broke the debug utilities. The symptom was in DEBUG_TRACE which would result in a compilation error due to a reference to the undefined class stlplus_debug_trace. This has been fixed.
The string formatting was being done by string-stream classes. I have rewritten it to use the more efficient portability/dprintf.
The CLI Parser loads default values from the Ini Manager. However, in previous versions, it was possible for a typo to mean that an option was ignored. I have now added error checking so that all variables in the relevant section of the Ini Manager are recognised as command-line options.
An extra error message has been added to messages/stlplus_messages.txt to report this error.
The Ini Manager now keeps track of which line in the Ini file a value comes from, so that error reporting can report the file name and line number of an error. The method ini_manager::variable_linenumber has been added for this purpose.
There was a bug in the quoting code for Windows command-lines which resulted in quote marks being added after every character in a string that needed quoting (i.e. contained special characters). This has been fixed.
The original STLplus was a single, rather large, library. It has now been split into 5 libraries. So STLplus is now referred to as a Library Collection, containing within it the STLplus containers, portability, persistence, subsystems and strings libraries.
All of the components in the STLplus library collection now have their own namespace - stlplus. Thus, the smart_ptr class is now stlplus::smart_ptr.
STLplus had it's own I/O subsystem for historical reasons. That was OK when I was the only user, but a bit much for new users to be expected to accept. I decided that TextIO had to go, but needed to wait for other major changes to justify a change that makes the library not backwards compatible.
STLplus version 3 does not have the TextIO subsystem any more - all functions using TextIO have been rewritten to use IOStream.
The string_arithmetic package was an anachronism - it was part of another project which seemed suitable for inclusion into STLplus, but now seems not to fit. So I have removed it.
The old name error_handler I think confused people as to the purpose of this component. Since STLplus 3 is not compatible with previous versions, I am taking this opportunity to rename it to better reflect its purpose - message_handler.
Iterators in the STL are badly designed - there are several major pitfalls with using them as a result of their design. However, I wanted STLplus to feel like the STL to make it easier to use, so felt I should use the iterator concept despite the problems associated with them. Some of the STLplus components therefore also use iterators.
In STLplus 2 I included some error checking for misuse of iterators. In STLplus 3 I have completed the job - so I have termed the STLplus iterators safe iterators.
The old STLplus used a technique for writing template functions that required external resources to be in the global namespace. This approach is not compatible with placing all of the STLplus into a separate namespace. All these functions have been rewritten to take an extra parameter so that those external resources are now passed as parameters.
This rewrite applies to the persistence library and the strings library.
STLplus 2 relied on overloading the functions dump and restore to implement persistence. However, because of the redesign of the functions discussed above, persistence functions are now passed as parameters to other persistence functions. To avoid ambiguity, all persistence functions now have unique names - dump_type and restore_type. See the persistence library for more information.
Note: the persistence file format is unchanged by this rewrite, so STLplus version 2 files can be read by version 3 and vice versa.
STLplus 2 string_utilities also relied on overloading the functions print and to_string to provide printing and formatting functions for whole data structures. However, because of the redesign of the functions discussed above, functions are now passed as parameters to other functions. To avoid ambiguity, all print and string-formatting functions now have unique names - print_type and type_to_string. See the strings library for more information.