Heavily improve documentation and add new container functions.

README.md

@@ -59,7 +59,7 @@ You can grab a single header out of the library [here](https://github.com/ThePhD
 
 ## Features
 
-- [Fastest in the land](http://sol2.readthedocs.io/en/latest/benchmarks.html) (see: sol2 graph and table entries).
+- [Fastest in the land](http://sol2.readthedocs.io/en/latest/benchmarks.html) (see: sol bar in graph).
 - Supports retrieval and setting of multiple types including `std::string` and `std::map/unordered_map`.
 - Lambda, function, and member function bindings are supported.
 - Intermediate type for checking if a variable exists.

docs/source/api/containers.rst

@@ -1,10 +1,14 @@
 containers
 ==========
-for handling ``std::vector/map`` and others
+for handling ``std::vector/map/set`` and others
--------------------------------------------
+-----------------------------------------------
 
 Sol2 automatically converts containers (detected using the ``sol::is_container<T>`` type trait, which simply looks for begin / end) to be a special kind of userdata with metatable on it. For Lua 5.2 and 5.3, this is extremely helpful as you can make typical containers behave like Lua tables without losing the actual container that they came from, as well as a small amount of indexing and other operations that behave properly given the table type.
 
+
+a complete example
+------------------
+
 Here's a complete working example of it working for Lua 5.3 and Lua 5.2, and how you can retrieve out the container in all versions:
 
 .. code-block:: cpp
@@ -20,7 +24,7 @@ Here's a complete working example of it working for Lua 5.3 and Lua 5.2, and how
 		lua.script(R"(
 		function f (x)
 			print('--- Calling f ---')
-			for k, v in ipairs(x) do
+			for k, v in pairs(x) do
 				print(k, v)
 			end
 		end
@@ -49,7 +53,24 @@ Here's a complete working example of it working for Lua 5.3 and Lua 5.2, and how
 		return 0;
 	}
 
-Note that this will not work well in 5.1, as it has explicit table checks and does not check metamethods, even when ``pairs`` or ``ipairs`` is passed a table. In that case, you will need to use a more manual iteration scheme.
+Note that this will not work well in 5.1, as it has explicit table checks and does not check metamethods, even when ``pairs`` or ``ipairs`` is passed a table. In that case, you will need to use a more manual iteration scheme or you will have to convert it to a table. In C++, you can use :doc:`sol::as_table<as_table>` when passing something to the library to get a table out of it.
+
+
+additional functions
+--------------------
+
+Based on the type pushed, a few additional functions are added as "member functions" (``self`` functions called with ``obj:func()`` or ``obj.func(obj)`` syntax) within a Lua script:
+
+* ``my_container:clear()``: This will call the underlying containers ``clear`` function.
+* ``my_container:add( key, value )`` or ``my_container:add( value )``: this will add to the end of the container, or if it is an associative or ordered container, simply put in an expected key-value pair into it.
+* ``my_contaner:insert( where, value )`` or ``my_contaner:insert( key, value )``: similar to add, but it only takes two arguments. In the case of ``std::vector`` and the like, the first argument is a ``where`` integer index. The second argument is the value. For associative containers, a key and value argument are expected.
+
+
+.. _container-detection:
+
+too-eager container detection?
+------------------------------
+
 
 If you have a type that has ``begin`` or ``end`` member functions but don't provide iterators, you can specialize ``sol::is_container<T>`` to be ``std::false_type``, and that will treat the type as a regular usertype and push it as a regular userdata:
 
@@ -69,4 +90,4 @@ If you have a type that has ``begin`` or ``end`` member functions but don't prov
 	namespace sol {
 		template <>
 		struct is_container<not_container> : std::false_type {};
-	}
+	}

docs/source/api/readonly.rst

@@ -65,4 +65,4 @@ If you are looking to make a read-only table, you need to go through a bit of a
 	    return 0;
 	}
 
-It is a verbose example, but it explains everything. Perhaps in the future ``sol2`` may feature a ``create_readonly_table`` abstraction for users!
+It is a verbose example, but it explains everything. Because the process is a bit involved and can have unexpected consequences for users that make their own tables, making read-only tables is something that we ask the users to do themselves with the above code, as getting the semantics right for the dozens of use cases would be tremendously difficult.

docs/source/api/state.rst

@@ -53,6 +53,7 @@ This function takes a number of :ref:`sol::lib<lib-enum>` as arguments and opens
 
 .. code-block:: cpp
 	:caption: function: script / script_file
+	:name: state-script-function
 
 	sol::function_result script(const std::string& code);
 	sol::function_result script_file(const std::string& filename);
@@ -89,8 +90,7 @@ These functions *load* the desired blob of either code that is in a string, or c
 	sol::protected_function_result do_string(const std::string& code);
 	sol::protected_function_result do_file(const std::string& filename);
 
-These functions *loads and performs* the desired blob of either code that is in a string, or code that comes from a filename, on the ``lua_State*``. It *will* run and returns a ``protected_function_result`` proxy that can be called to actually run the code, turned into a ``sol::function``, a ``sol::protected_function``, or some other abstraction.
+These functions *loads and performs* the desired blob of either code that is in a string, or code that comes from a filename, on the ``lua_State*``. It *will* run, and then return a ``protected_function_result`` proxy that can be examined for either an error or the return value.
-
 
 .. code-block:: cpp
 	:caption: function: global table / registry table
@@ -107,7 +107,7 @@ Get either the global table or the Lua registry as a :doc:`sol::table<table>`, w
 
 	void set_panic(lua_CFunction panic);
 
-Overrides the panic function Lua calls when something unrecoverable or unexpected happens in the Lua VM. Must be a function of the that matches the ``int(*)(lua_State*)`` function signature.
+Overrides the panic function Lua calls when something unrecoverable or unexpected happens in the Lua VM. Must be a function of the that matches the ``int(lua_State*)`` function signature.
 
 .. code-block:: cpp
 	:caption: function: make a table

docs/source/errors.rst

@@ -0,0 +1,35 @@
+errors
+======
+how to handle exceptions or other errors 
+----------------------------------------
+
+Here is some advice and some tricks to use when dealing with thrown exceptions, error conditions and the like in Sol.
+
+Catch and CRASH!
+----------------
+
+By default, Sol will add a ``default_at_panic`` handler. If exceptions are not turned off, this handler will throw to allow the user a chance to recover. However, in almost all cases, when Lua calls ``lua_atpanic`` and hits this function, it means that something *irreversibly wrong* occured in your code or the Lua code and the VM is in an unpredictable or dead state. Catching an error thrown from the default handler and then proceeding as if things are cleaned up or okay is NOT the best idea. Unexpected bugs in optimized and release mode builds can result, among other serious issues.
+
+
+It is preferred if you catch an error that you log what happened, terminate the Lua VM as soon as possible, and then crash if your application cannot handle spinning up a new Lua state. Catching can be done, but you should understand the risks of what you're doing when you do it.
+
+
+Destructors and Safety
+----------------------
+
+Another issue is that Lua is a C API. It uses ``setjmp`` and ``longjmp`` to jump out of code when an error occurs. This means it will ignore destructors in your code if you use the library or the underlying Lua VM improperly. To solve this issue, build Lua as C++. When a Lua VM error occurs and ``lua_error`` is triggered, it raises it as an exception which will provoke proper unwinding semantics.
+
+
+Protected Functions and Access
+------------------------------
+
+By default, :doc:`sol::function<api/function>` assumes the code ran just fine and there are no problems. :ref:`sol::state(_view)::script(_file)<state-script-function>` also assumes that code ran just fine. Use :doc:`sol::protected_function<api/protected_function>` to have function access where you can check if things worked out. Use :doc:`sol::optional<api/optional>` to get a value safely from Lua. Use :ref:`sol::state(_view)::do_string/do_file/load/load_file<state-do-code>` to safely load and get results from a script. The defaults are provided to be simple and fast with thrown exceptions to violently crash the VM in case things go wrong.
+
+Raw Functions
+-------------
+
+When you push a function into Lua using Sol using any methods and that function exactly matches the signature ``int( lua_State* );``, it will be treated as a *raw C function*. This means that the usual exception trampoline Sol wraps your other function calls in will not be present. You will be responsible for catching exceptions and handling them before they explode into the C API (and potentially destroy your code). Sol in all other cases adds an exception-handling trampoline that turns exceptions into Lua errors that can be caught by the above-mentioned protected functions and accessors.
+
+.. warning::
+	
+	Do NOT assume that building Lua as C++ will allow you to throw directly from a raw function. If an exception is raised and it bubbles into the Lua framework, even if you compile as C++, Lua does not recognize exceptions other than the ones that it uses with ``lua_error``. In other words, it will return some completely bogus result, potentially leave your Lua stack thrashed, and the rest of your VM *can* be in a semi-trashed state. Please avoid this!

docs/source/index.rst

@@ -32,8 +32,10 @@ get going:
 	
 	tutorial/all-the-things
 	tutorial/tutorial-top
+	errors
 	features
 	usertypes
+	traits
 	api/api-top
 	mentions
 	benchmarks

docs/source/traits.rst

@@ -0,0 +1,15 @@
+customization traits
+====================
+
+These are customization points within the library to help you make sol2 work for the types in your framework and types. 
+
+To learn more about various customizable traits, visit:
+
+* :ref:`containers detection trait<container-detection>`
+	- This is how to work with containers when you have an compiler error when serializing a type that has ``begin`` and ``end`` functions but isn't exactly a container.
+* :doc:`unique usertype (custom pointer) traits<api/unique_usertype_traits>`
+	- This is how to deal with unique usertypes, e.g. ``boost::shared_ptr``, reference-counted pointers, etc.
+	- Useful for custom pointers from all sorts of frameworks or handle types that employ very specific kinds of destruction semantics and access.
+* :doc:`customization point tutorial<tutorial/customization>`
+	- This is how to customize a type to work with sol2.
+	- Can be used for specializations to push strings and other class types that are not natively ``std::string`` or ``const char*``.

docs/source/usertypes.rst

@@ -5,12 +5,12 @@ Perhaps the most powerful feature of sol2, ``usertypes`` are the way sol2 and C+
 
 To learn more about usertypes, visit:
 
-* :doc:`tutorial<tutorial/cxx-in-lua>`
+* :doc:`the basic tutorial<tutorial/cxx-in-lua>`
 * :doc:`customization point tutorial<tutorial/customization>`
 * :doc:`api documentation<api/usertype>`
 * :doc:`memory documentation<api/usertype_memory>`
 
-There are also some notes about guarantees you can find about usertypes, and their associated userdata, below:
+The examples folder also has a number of really great examples for you to see. There are also some notes about guarantees you can find about usertypes, and their associated userdata, below:
 
 * You can push types classified as userdata before you register a usertype.
 	- You can register a usertype with the Lua runtime at any time sol2

sol/container_usertype_metatable.hpp

@@ -41,6 +41,19 @@ namespace sol {
 			static const bool value = sizeof(test<T>(0)) == sizeof(char);
 		};
 
+		template <typename T>
+		struct has_push_back {
+		private:
+			typedef std::array<char, 1> one;
+			typedef std::array<char, 2> two;
+
+			template <typename C> static one test(decltype(&C::push_back));
+			template <typename C> static two test(...);
+
+		public:
+			static const bool value = sizeof(test<T>(0)) == sizeof(char);
+		};
+
 		template <typename T>
 		T& get_first(const T& t) {
 			return std::forward<T>(t);
@@ -183,13 +196,23 @@ namespace sol {
 			return stack::push(L, src.size());
 		}
 
-#if 0
+		static int real_add_call(std::true_type, lua_State*L) {
-		static int real_push_back_call(lua_State*L) {
 			auto& src = get_src(L);
 			src.push_back(stack::get<V>(L, 2));
 			return 0;
 		}
 
+		static int real_add_call(std::false_type, lua_State*L) {
+			using std::cend;
+			auto& src = get_src(L);
+			src.insert(cend(src), stack::get<V>(L, 2));
+			return 0;
+		}
+
+		static int real_add_call(lua_State*L) {
+			return real_add_call(std::integral_constant<bool, detail::has_push_back<T>::value>(), L);
+		}
+
 		static int real_insert_call(lua_State*L) {
 			using std::begin;
 			auto& src = get_src(L);
@@ -197,14 +220,23 @@ namespace sol {
 			return 0;
 		}
 
-		static int push_back_call(lua_State*L) {
+		static int real_clear_call(lua_State*L) {
-			return detail::static_trampoline<(&real_length_call)>(L);
+			auto& src = get_src(L);
+			src.clear();
+			return 0;
+		}
+
+		static int add_call(lua_State*L) {
+			return detail::static_trampoline<(&real_add_call)>(L);
 		}
 
 		static int insert_call(lua_State*L) {
 			return detail::static_trampoline<(&real_insert_call)>(L);
 		}
-#endif // Sometime later, in a distant universe...
+
+		static int clear_call(lua_State*L) {
+			return detail::static_trampoline<(&real_clear_call)>(L);
+		}
 
 		static int length_call(lua_State*L) {
 			return detail::static_trampoline<(&real_length_call)>(L);
@@ -320,6 +352,28 @@ namespace sol {
 			return stack::push(L, src.size());
 		}
 
+		static int real_insert_call(lua_State*L) {
+			return real_new_index_call(L);
+		}
+
+		static int real_clear_call(lua_State*L) {
+			auto& src = get_src(L);
+			src.clear();
+			return 0;
+		}
+
+		static int add_call(lua_State*L) {
+			return detail::static_trampoline<(&real_insert_call)>(L);
+		}
+
+		static int insert_call(lua_State*L) {
+			return detail::static_trampoline<(&real_insert_call)>(L);
+		}
+
+		static int clear_call(lua_State*L) {
+			return detail::static_trampoline<(&real_clear_call)>(L);
+		}
+
 		static int length_call(lua_State*L) {
 			return detail::static_trampoline<(&real_length_call)>(L);
 		}
@@ -344,61 +398,79 @@ namespace sol {
 	namespace stack {
 		namespace stack_detail {
 			template <typename T>
-			inline const auto& container_metatable() {
+			inline auto container_metatable() {
-				typedef container_usertype_metatable<std::remove_pointer_t<T>> cumt;
+				typedef container_usertype_metatable<std::remove_pointer_t<T>> meta_cumt;
-				static const luaL_Reg reg[] = {
+				std::array<luaL_Reg, 7> reg = { {
-					{ "__index", &cumt::index_call },
+					{ "__index", &meta_cumt::index_call },
-					{ "__newindex", &cumt::new_index_call },
+					{ "__newindex", &meta_cumt::new_index_call },
-					{ "__pairs", &cumt::pairs_call },
+					{ "__pairs", &meta_cumt::pairs_call },
-					{ "__ipairs", &cumt::pairs_call },
+					{ "__ipairs", &meta_cumt::pairs_call },
-					{ "__len", &cumt::length_call },
+					{ "__len", &meta_cumt::length_call },
-					{ std::is_pointer<T>::value ? nullptr : "__gc", std::is_pointer<T>::value ? nullptr : &detail::usertype_alloc_destroy<T> },
+					std::is_pointer<T>::value ? luaL_Reg{ nullptr, nullptr } : luaL_Reg{ "__gc", &detail::usertype_alloc_destroy<T> },
 					{ nullptr, nullptr }
-				};
+				} };
 				return reg;
 			}
-		}
 
-		template<typename T>
+			template <typename T>
-		struct pusher<T, std::enable_if_t<meta::all<is_container<T>, meta::neg<meta::any<std::is_base_of<reference, T>, std::is_base_of<stack_reference, T>>>>::value>> {
+			inline auto container_metatable_behind() {
-			typedef container_usertype_metatable<T> cumt;
+				typedef container_usertype_metatable<std::remove_pointer_t<T>> meta_cumt;
-			static int push(lua_State* L, const T& cont) {
+				std::array<luaL_Reg, 5> reg = { {
-				auto fx = [&L]() {
+					{ "__index", &meta_cumt::index_call },
-					const char* metakey = &usertype_traits<T>::metatable()[0];
+					{ "clear", &meta_cumt::clear_call },
+					{ "insert", &meta_cumt::insert_call },
+					{ "add", &meta_cumt::add_call },
+					{ nullptr, nullptr }
+				} };
+				return reg;
+			}
+
+			template <typename T>
+			struct metatable_setup {
+				lua_State* L;
+
+				metatable_setup(lua_State* L) : L(L) {}
+
+				void operator()() {
+					static const auto reg = container_metatable<T>();
+					static const auto containerreg = container_metatable_behind<T>();
+					static const char* metakey = &usertype_traits<T>::metatable()[0];
+					
 					if (luaL_newmetatable(L, metakey) == 1) {
-						const auto& reg = stack_detail::container_metatable<T>();
+						stack_reference metatable(L, -1);
-						luaL_setfuncs(L, reg, 0);
+						luaL_setfuncs(L, reg.data(), 0);
+
+						lua_createtable(L, 0, static_cast<int>(containerreg.size()));
+						stack_reference metabehind(L, -1);
+						luaL_setfuncs(L, containerreg.data(), 0);
+						stack::set_field(L, sol::meta_function::index, metabehind, metabehind.stack_index());
+
+						stack::set_field(L, metatable_key, metabehind, metatable.stack_index());
+						stack::set_field(L, sol::meta_function::index, metabehind, metatable.stack_index());
+						metabehind.pop();
 					}
 					lua_setmetatable(L, -2);
-				};
+				}
+			};
+		}
+		
+		template<typename T>
+		struct pusher<T, std::enable_if_t<meta::all<is_container<T>, meta::neg<meta::any<std::is_base_of<reference, meta::unqualified_t<T>>, std::is_base_of<stack_reference, meta::unqualified_t<T>>>>>::value>> {
+			static int push(lua_State* L, const T& cont) {
+				stack_detail::metatable_setup<T> fx(L);
 				return pusher<detail::as_value_tag<T>>{}.push_fx(L, fx, cont);
 			}
 
 			static int push(lua_State* L, T&& cont) {
-				auto fx = [&L]() {
+				stack_detail::metatable_setup<T> fx(L);
-					const char* metakey = &usertype_traits<T>::metatable()[0];
-					if (luaL_newmetatable(L, metakey) == 1) {
-						const auto& reg = stack_detail::container_metatable<T>();
-						luaL_setfuncs(L, reg, 0);
-					}
-					lua_setmetatable(L, -2);
-				};
 				return pusher<detail::as_value_tag<T>>{}.push_fx(L, fx, std::move(cont));
 			}
 		};
 
 		template<typename T>
 		struct pusher<T*, std::enable_if_t<meta::all<is_container<T>, meta::neg<meta::any<std::is_base_of<reference, meta::unqualified_t<T>>, std::is_base_of<stack_reference, meta::unqualified_t<T>>>>>::value>> {
-			typedef container_usertype_metatable<T> cumt;
 			static int push(lua_State* L, T* cont) {
-				auto fx = [&L]() {
+				stack_detail::metatable_setup<T*> fx(L);
-					const char* metakey = &usertype_traits<meta::unqualified_t<T>*>::metatable()[0];
-					if (luaL_newmetatable(L, metakey) == 1) {
-						const auto& reg = stack_detail::container_metatable<T*>();
-						luaL_setfuncs(L, reg, 0);
-					}
-					lua_setmetatable(L, -2);
-				};
 				return pusher<detail::as_pointer_tag<T>>{}.push_fx(L, fx, cont);
 			}
 		};

sol/types.hpp

@@ -400,7 +400,7 @@ namespace sol {
 		"__newindex",
 		"__mode",
 		"__call",
-		"__metatable",
+		"__mt",
 		"__tostring",
 		"__len",
 		"__unm",

test_containers.cpp

@@ -273,6 +273,57 @@ TEST_CASE("containers/arbitrary-creation", "userdata and tables should be usable
 	REQUIRE(c.get<std::string>("project") == "sol");
 }
 
+TEST_CASE("containers/extra-functions", "make sure the manipulation functions are present and usable and working across various container types") {
+	sol::state lua;
+	lua.open_libraries();
+
+	lua.script(R"(
+function g (x)
+	x:add(20)
+end
+
+function h (x)
+	x:add(20, 40)
+end
+
+function i (x)
+	x:clear()
+end
+)");
+
+	// Have the function we 
+	// just defined in Lua
+	sol::function g = lua["g"];
+	sol::function h = lua["h"];
+	sol::function i = lua["i"];
+
+	// Set a global variable called 
+	// "arr" to be a vector of 5 lements
+	lua["arr"] = std::vector<int>{ 2, 4, 6, 8, 10 };
+	lua["map"] = std::unordered_map<int, int>{ { 1 , 2 },{ 2, 4 },{ 3, 6 },{ 4, 8 },{ 5, 10 } };
+	lua["set"] = std::set<int>{ 2, 4, 6, 8, 10 };
+	std::vector<int>& arr = lua["arr"];
+	std::map<int, int>& map = lua["map"];
+	std::set<int>& set = lua["set"];
+	REQUIRE(arr.size() == 5);
+	REQUIRE(map.size() == 5);
+	REQUIRE(set.size() == 4);
+
+	g(lua["set"]);
+	g(lua["arr"]);
+	h(lua["map"]);
+	REQUIRE(arr.size() == 6);
+	REQUIRE(map.size() == 6);
+	REQUIRE(set.size() == 6);
+
+	i(lua["arr"]);
+	i(lua["map"]);
+	i(lua["set"]);	
+	REQUIRE(arr.empty());
+	REQUIRE(map.empty());
+	REQUIRE(set.empty());
+}
+
 TEST_CASE("containers/usertype-transparency", "Make sure containers pass their arguments through transparently and push the results as references, not new values") {
 	class A {
 	public: