This post written by Xiang Fan and Andrew Pardoe
Last December we blogged about partial Expression SFINAE support in VS 2015 Update 1. Some of the things we heard from you after that post are that you wanted better Expression SFINAE support, especially for popular libraries such as Boost and Range-v3. These libraries have been our focus over the last few months.
What is SFINAE?
As a reminder, SFINAE is an acronym for ‘Substitution Failure Is Not An Error’. The idea is that when the compiler tries to specialize a function template during overload resolution, it is ok if the specialization fails as long as there are other valid candidates. C++11 introduced features like decltype and constexpr, and it is more common to have expressions during the template argument deduction and substitution process. The C++ standards committee clarified the SFINAE rules for expressions in C++11.
Boost support
The big news with this update is that Boost now compiles correctly with MSVC without defining the macro BOOST_NO_SFINAE_EXPR. We worked with the Boost maintainers and community to make sure we got it correct. There are still libraries and open source projects that build on top of Boost that have blocking issues. Please let us know if you run into any issues. You can email us at visualcpp@microsoft.com or use any of the feedback channels below.
There are still three failing tests in the Boost test suite when compiling with MSVC. Two test failures relate to using the noexcept operator in a template type argument. We currently parse it immediately, even in a dependent context. One test failure is a Boost library issue that we’ve reported to the maintainers.
Range-v3 support
While we’ve been testing with Range-v3, our progress isn’t as complete as it is with Boost. We went through all range-v3 tests and identified various issues in it. We logged over 50 bugs internally to track our progress and have fixed about half of these bugs. Issues remain in many different feature areas, such as name lookup, friend functions, variadic templates, alias templates, default template arguments, constexpr and noexcept. There are other issues mentioned in the What’s Remaining section below.
Improvements since Update 1
Our implementation is still partial , but we’re almost done. We’ve made a lot of progress since Update 1. First, though let’s talk about why it’s so hard to get Expression SFINAE working in the MSVC compiler.
MSVC’s token stream parser
The MSVC compiler has been around since before C++ had templates–we’re now working around design decisions that once made sense. MSVC traditionally took a token stream-based approach to parsing templates. When we encounter a template in your code we capture its body as a sequence of tokens without any attempt to understand what the tokens mean. Storing the body as a stream of tokens makes analysis of trailing return types containing decltype-specifiers imperfect, especially in SFINAE contexts.
We have now implemented a recursive-descent parser that generates high level unbound trees for expressions and employed this to analyze the expression argument of decltype in a much more precise way, allowing a better implementation of Expression SFINAE. The recursive descent parser is a work in progress; currently, it can parse only C++ expressions but we’re going to soon expand it to parse the entire C++ syntax and make it the basis for implementing features such as two-phase name lookup. These features have been almost impossible to implement with the token stream-based parser. As work proceeds, the remaining gaps in Expression SFINAE will also be filled.
If you’d like to read more about the changes we’re making to the parser you can find more in this blog post: Rejuvenating the Microsoft C/C++ Compiler.
What’s working now?
Because we’re now generating parse trees for decltype expressions a number of patterns work correctly in Update 3.
- We’ve implemented checking for dependent expression using the new parse tree in the compiler. That fixes this Connect issue reported for a failure compiling Chromium.
- We’ve implemented ability to distinguish different expressions inside decltype using parse tree. Here’s an example simplified from the Boost thread library:
template<class T> struct remove_reference { typedef T type; }; template<class T> inline T&& forward(typename remove_reference<T>::type& t) { return static_cast<T&&>(t); } template<class T> inline T&& forward(typename remove_reference<T>::type&& t) { return static_cast<T&&>(t); } template <class Fp, class A0, class ...Args> inline auto invoke(Fp && f, A0 && a0, Args && ...args) -> decltype((forward<A0>(a0).*f)(forward<Args>(args)...)) { return (forward<A0>(a0).*f)(forward<Args>(args)...); } template <class Fp, class A0, class ...Args> inline auto invoke(Fp && f, A0 && a0, Args && ...args) -> decltype(((*forward<A0>(a0)).*f)(forward<Args>(args)...)) { return ((*forward(a0)).*f)(forward(args)...); } - A couple of test cases simplified from Range-v3 now work.
int f(int *); namespace N { template<typename T> T val(); template<typename T> using void_t = void; template<typename T, typename = void> struct trait {}; template<typename T> struct trait<T, void_t<decltype(f(val<T>()))>> { typedef decltype(f(val<T>())) type; }; } N::trait<int *>::type t1; struct S { template<typename T> static T val(); template<typename T> using void_t = void; template<typename T, typename = void> struct trait {}; template<typename T> struct trait<T, void_t<decltype(f(val<T>()))>> { typedef decltype(f(val<T>())) type; }; }; S::trait<int *>::type t2; - Also, this example:
int g; template<typename T> using void_t = void; template<typename T, typename = void> struct S1 {}; template<typename T> struct S1<T, void_t<decltype(g + T{}) >> {}; struct S2 { int *g; auto f() -> decltype(S1<int>()); };
What remains to be done?
constexprused inside template arguments:
Whenever we see a function call in a template non-type argument, we parse and do semantic analysis on it immediately. This will prevent its use in a dependent context.constexpris commonly used as template non-type argument in range-v3.#include <type_traits> #if defined(WORKAROUND1) // Alternative for type_trait. template<typename T, std::enable_if_t<std::is_pointer<T>::value> * = nullptr> #elif defined(WORKAROUND2) // If the constexpr is always valid, it can be moved into a helper class template<typename T> struct helper { static const bool value = T{}; }; template<typename T, std::enable_if_t<helper<std::is_pointer<T>>::value> * = nullptr> #else // VC does semantic analysis on 'std::is_pointer<T>{}' prematurely // and gives error on the function declaration // error C2512: 'std::is_pointer<_Ty>': no appropriate default constructor available template<typename T, std::enable_if_t<std::is_pointer<T>{}> * = nullptr> #endif short f(const T &); char f(...); int main() { int *p = nullptr; static_assert(sizeof(f(0)) == sizeof(char), "fail"); static_assert(sizeof(f(p)) == sizeof(short), "fail"); }- Decltype and dependent expressions inside alias templates:
If the definition of alias template contains decltype or dependent expression, we will capture its tokens and re-parse it later. Because the tokens don’t resolve the name (they only store the identifier), sometimes the alias specialization will find the wrong symbol during re-parse and you may get incorrect SFINAE result. For example, these two code snippets:// General case template<typename> struct S { using type = int; }; template<typename T> using type1 = decltype(S<T>{}); template<typename T> using type2 = typename type1<T>::type; type2<int> i;// This case impacts SFINAE. Simplified from range-v3 template<typename> struct S1 {}; template<typename T> T declval(); void f(int *); #ifdef WORKAROUND template <typename T> using void_t = void; template <class T, class V = void> struct helper {}; template <class T> struct helper<T, void_t<decltype(f(declval<T>()))>> { typedef decltype(f(declval<T>())) type; }; template<typename T> using S2 = typename helper<T>::type; #else template<typename T> using S2 = decltype(f(declval<T>())); #endif template<typename U, typename T, S1<S2<U>> * = nullptr> short g(U, T); char g(...); void h() { // we are supposed to use 'std::nullptr_t' to specialize S2<U>, // but we incorrectly use 'int' during re-parsing static_assert(sizeof(g(nullptr, 1)) == sizeof(short), "fail"); // we are supposed to use 'int' to specialize S2<U>, // but we incorrectly use 'std::nullptr_t' during re-parsing static_assert(sizeof(g(1, nullptr)) == sizeof(char), "fail"); } - Pack expansions used inside decltype expressions:We are currently not able to detect pack expansions inside
decltype, so it won’t be expanded correctly in some cases. You may get incorrect SFINAE result because of this.// Example from range-v3: #ifdef WORKAROUND template<typename T> struct helper1 {}; template<typename Fun2, typename ...Args2> struct helper2 { // pack expansion works for function template template<typename Fun, typename ...Args> static short f(helper1<decltype(val<Fun>()(val<Args>()...))> * = 0); template<typename Fun, typename ...Args> static char f(...); static const bool value = sizeof(helper2::f<Fun2, Args2...>(0)) == sizeof(short); }; template<bool, typename Fun, typename ...Args> struct helper { }; template<typename Fun, typename ...Args> struct helper<true, Fun, Args...> { typedef decltype(val<Fun>()(val<Args>()...)) type; }; template<typename Fun, typename ...Args> using result_t = typename helper<helper2<Fun, Args...>::value, Fun, Args...>::type; #else template<typename Fun, typename ...Args> using result_t = decltype(val<Fun>()(val<Args>()...)); #endif
Send us feedback!
As always, we want to hear from you. If you know of any issues with Expression SFINAE not covered in this post, please send us feedback through Send us your feedback in the comments below, through Connect, or by sending us a mail at visualcpp@microsoft.com. Thank you!
Try out the new compiler today! The official VS announcement about the VS 2015 Update 3 RC release is here: https://blogs.msdn.microsoft.com/visualstudio/2016/06/07/visual-studio-2015-update-3-rc/