Message ID | 20161126070354.141764-11-wangnan0@huawei.com |
---|---|
State | Accepted |
Commit | 00b86691c77c6576861b82a3cfe4d609800758fe |
Headers | show |
On Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan wrote: > Add basic clang support in clang.cpp and test__clang() testcase. The > first testcase checks if builtin clang is able to generate LLVM IR. > > tests/clang.c is a proxy. Real testcase resides in > utils/c++/clang-test.cpp in c++ and exports C interface to perf test > subsystem. > > Test result: > > $ perf test -v clang > 51: Test builtin clang support : > 51.1: Test builtin clang compile C source to IR : > --- start --- > test child forked, pid 13215 > test child finished with 0 > ---- end ---- > Test builtin clang support subtest 0: Ok > > Signed-off-by: Wang Nan <wangnan0@huawei.com> ... > +static CompilerInvocation * > +createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags) > +{ > + llvm::opt::ArgStringList CCArgs { > + "-cc1", > + "-triple", "bpf-pc-linux", > + "-fsyntax-only", > + "-ferror-limit", "19", > + "-fmessage-length", "127", why such limits? > + "-O2", > + "-nostdsysteminc", > + "-nobuiltininc", > + "-vectorize-loops", > + "-vectorize-slp", why above two flags are needed? > + "-Wno-unused-value", > + "-Wno-pointer-sign", these two -Wno makes sense. please add the comment to explain the reasons.
On 2016/11/27 1:17, Alexei Starovoitov wrote: > On Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan wrote: >> Add basic clang support in clang.cpp and test__clang() testcase. The >> first testcase checks if builtin clang is able to generate LLVM IR. >> >> tests/clang.c is a proxy. Real testcase resides in >> utils/c++/clang-test.cpp in c++ and exports C interface to perf test >> subsystem. >> >> Test result: >> >> $ perf test -v clang >> 51: Test builtin clang support : >> 51.1: Test builtin clang compile C source to IR : >> --- start --- >> test child forked, pid 13215 >> test child finished with 0 >> ---- end ---- >> Test builtin clang support subtest 0: Ok >> >> Signed-off-by: Wang Nan <wangnan0@huawei.com> > ... >> +static CompilerInvocation * >> +createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags) >> +{ >> + llvm::opt::ArgStringList CCArgs { >> + "-cc1", >> + "-triple", "bpf-pc-linux", >> + "-fsyntax-only", >> + "-ferror-limit", "19", >> + "-fmessage-length", "127", > why such limits? > >> + "-O2", >> + "-nostdsysteminc", >> + "-nobuiltininc", >> + "-vectorize-loops", >> + "-vectorize-slp", Thank you for pointing these out. These arguments are get by analysising the clang example: https://llvm.org/svn/llvm-project/cfe/branches/ggreif/CallInst-operands/examples/clang-interpreter/main.cpp The above example create a C compiler using clang::driver::Driver (bcc also uses driver). I form the argument list according to arglist the driver created for its CI, and leaves arguments I'm not quite sure unchanged. > why above two flags are needed? > >> + "-Wno-unused-value", >> + "-Wno-pointer-sign", > these two -Wno makes sense. please add the comment to explain the reasons. > They are inherited from samples/bpf/Makefile to suppress some warning when include kernel headers. Thank you.
Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu: > Add basic clang support in clang.cpp and test__clang() testcase. The > first testcase checks if builtin clang is able to generate LLVM IR. > > tests/clang.c is a proxy. Real testcase resides in > utils/c++/clang-test.cpp in c++ and exports C interface to perf test > subsystem. > > Test result: > > $ perf test -v clang > 51: Test builtin clang support : > 51.1: Test builtin clang compile C source to IR : > --- start --- > test child forked, pid 13215 > test child finished with 0 > ---- end ---- > Test builtin clang support subtest 0: Ok While testing this I noticed that the perf binary got huge, can't this be done in some other way, i.e. using dynamic library? [root@jouet ~]# size /tmp/perf text data bss dec hex filename 75333688 1421584 23962176 100717448 600d388 /tmp/perf [root@jouet ~]# I've built it with this: make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin the resulting binary: [acme@jouet linux]$ [acme@jouet linux]$ ls -la ~/bin/perf -rwxr-xr-x. 2 acme acme 131689136 Dec 2 12:31 /home/acme/bin/perf [acme@jouet linux]$ ls -lah ~/bin/perf -rwxr-xr-x. 2 acme acme 126M Dec 2 12:31 /home/acme/bin/perf [acme@jouet linux]$ Huge, after stripping it: [acme@jouet linux]$ ls -la /tmp/perf -rwxr-xr-x. 1 root root 76759056 Dec 2 12:33 /tmp/perf [acme@jouet linux]$ ls -lah /tmp/perf -rwxr-xr-x. 1 root root 74M Dec 2 12:33 /tmp/perf [acme@jouet linux]$ Still huge :-\ Ok, this needs explicitely enabling, but then, for distros, if we want this feature to be enabled... I think we should try to help distributions to ship a basic perf binary in a base package that doesn't brings many libraries, like was done for the gtk UI, and also we should start providing RPM specfiles and debian (and others, hopefully contributed by other people with interest in other distros), to help in getting a sane package split, i.e. want llvm/bpf? install the 'perf-ebpf' rpm/deb/whatever package and incur in these costs. But being a n00b on llvm/clang libraries, etc, my question goes back to: can we have this using a libllvm.so or libclang.so dynamic libraries? See this, out of hand :-\ Tell me I'm doing something wrong :-) [acme@jouet linux]$ cat /etc/fedora-release Fedora release 25 (Twenty Five) [acme@jouet linux]$ rpm -qa | egrep llvm\|clang llvm-libs-3.8.0-1.fc25.x86_64 [acme@jouet linux]$ [acme@jouet linux]$ nm ~/bin/perf | grep -i clang | wc -l 204757 [acme@jouet linux]$ nm ~/bin/perf | grep -i llvm | wc -l 234036 [acme@jouet linux]$ nm ~/bin/perf | grep -i clang | tail 00000000011741ba t _ZZZN5clang7CodeGen15CodeGenFunction20EmitOMPTaskDirectiveERKNS_16OMPTaskDirectiveEENKUlRS1_E_clES5_ENKUlvE_clEv 0000000001169fee t _ZZZN5clang7CodeGen15CodeGenFunction25EmitOMPFirstprivateClauseERKNS_22OMPExecutableDirectiveERNS1_15OMPPrivateScopeEENKUlvE_clEvENKUlNS0_7AddressES8_E_clES8_S8_ 00000000038b83e0 r _ZZZN5clang9ASTReader24FindExternalLexicalDeclsEPKNS_11DeclContextEN4llvm12function_refIFbNS_4Decl4KindEEEERNS4_15SmallVectorImplIPS6_EEENKUlPNS_13serialization10ModuleFileENS4_8ArrayRefINS4_7support6detail31packed_endian_specific_integralIjLNSI_10endiannessE2ELm1EEEEEE_clESG_SN_E19__PRETTY_FUNCTION__ 0000000003934140 r _ZZZN5clang9ASTWriter12WriteASTCoreERNS_4SemaEN4llvm9StringRefERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPNS_6ModuleEENKUljbE0_clEjbE19__PRETTY_FUNCTION__ 00000000039319e0 r _ZZZN5clang9ASTWriter12WriteASTCoreERNS_4SemaEN4llvm9StringRefERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEPNS_6ModuleEENKUlPNS_4DeclENS_13serialization17PredefinedDeclIDsEE_clESG_SI_E19__PRETTY_FUNCTION__ 000000000393b700 r _ZZZN5clang9ASTWriter17GetOrCreateTypeIDENS_8QualTypeEENKUlS1_E_clES1_E19__PRETTY_FUNCTION__ 000000000393b880 r _ZZZNK5clang9ASTWriter9getTypeIDENS_8QualTypeEENKUlS1_E_clES1_E19__PRETTY_FUNCTION__ 0000000002176480 t _ZZZZN12_GLOBAL__N_19ASTDumper11dumpLookupsEPKN5clang11DeclContextEbENKUlvE_clEvENKUlvE_clEvENKUlvE_clEv 000000000144ef0a t _ZZZZN5clang7CodeGen15CGOpenMPRuntime13emitReductionERNS0_15CodeGenFunctionENS_14SourceLocationEN4llvm8ArrayRefIPKNS_4ExprEEESA_SA_SA_bbENKUlS3_S9_S9_S9_E1_clES3_S9_S9_S9_ENKUlNS0_6RValueEE_clESC_ENKUlvE_clEv 000000000217640c t _ZZZZZN12_GLOBAL__N_19ASTDumper11dumpLookupsEPKN5clang11DeclContextEbENKUlvE_clEvENKUlvE_clEvENKUlvE_clEvENKUlPNS1_4DeclEE_clES9_ [acme@jouet linux]$ Thanks, - Arnaldo
On 2016/12/2 23:44, Arnaldo Carvalho de Melo wrote: > Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu: >> Add basic clang support in clang.cpp and test__clang() testcase. The >> first testcase checks if builtin clang is able to generate LLVM IR. >> >> tests/clang.c is a proxy. Real testcase resides in >> utils/c++/clang-test.cpp in c++ and exports C interface to perf test >> subsystem. >> >> Test result: >> >> $ perf test -v clang >> 51: Test builtin clang support : >> 51.1: Test builtin clang compile C source to IR : >> --- start --- >> test child forked, pid 13215 >> test child finished with 0 >> ---- end ---- >> Test builtin clang support subtest 0: Ok > While testing this I noticed that the perf binary got huge, can't this > be done in some other way, i.e. using dynamic library? I intentionally use statically linking because it is good for smartphone: we can simply 'adb push' a statically linked perf to Android. The resulting ELF executable would be even larger if LLVM is built with default setting. In my setting the resuling 'perf' is less than 60MB: $ ls -s ~/perf -h 58M /home/wn/perf $ size ~/perf text data bss dec hex filename 56931273 2950808 24108632 83990713 50198b9 /home/wn/perf It is reasonable for me. I think using dynamic clang and llvm libraries is possible but I never tried it before. It depend on LLVM compiling. I think if distro provides shared libraries then perf can utilize them automatically. Let me check it today. About the file size, I discussed with Alexei, he taught me a lot on it. Maybe he or his ioVisor ex-colleagues can provide some new idea? Thank you.
On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote: > Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu: > > Add basic clang support in clang.cpp and test__clang() testcase. The > > first testcase checks if builtin clang is able to generate LLVM IR. > > > > tests/clang.c is a proxy. Real testcase resides in > > utils/c++/clang-test.cpp in c++ and exports C interface to perf test > > subsystem. > > > > Test result: > > > > $ perf test -v clang > > 51: Test builtin clang support : > > 51.1: Test builtin clang compile C source to IR : > > --- start --- > > test child forked, pid 13215 > > test child finished with 0 > > ---- end ---- > > Test builtin clang support subtest 0: Ok > > While testing this I noticed that the perf binary got huge, can't this > be done in some other way, i.e. using dynamic library? > > [root@jouet ~]# size /tmp/perf > text data bss dec hex filename > 75333688 1421584 23962176 100717448 600d388 /tmp/perf > [root@jouet ~]# > > I've built it with this: > > make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin > > the resulting binary: > > [acme@jouet linux]$ > [acme@jouet linux]$ ls -la ~/bin/perf > -rwxr-xr-x. 2 acme acme 131689136 Dec 2 12:31 /home/acme/bin/perf > [acme@jouet linux]$ ls -lah ~/bin/perf > -rwxr-xr-x. 2 acme acme 126M Dec 2 12:31 /home/acme/bin/perf > [acme@jouet linux]$ > > Huge, after stripping it: > > [acme@jouet linux]$ ls -la /tmp/perf > -rwxr-xr-x. 1 root root 76759056 Dec 2 12:33 /tmp/perf > [acme@jouet linux]$ ls -lah /tmp/perf > -rwxr-xr-x. 1 root root 74M Dec 2 12:33 /tmp/perf > [acme@jouet linux]$ > > Still huge :-\ yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode. Try llvm-config --build-mode --assertion-mode it should be Release OFF > But being a n00b on llvm/clang libraries, etc, my question goes back to: > can we have this using a libllvm.so or libclang.so dynamic libraries? that can also work. The reason we build iovisor/bcc into single binary is to ease operational headache.
Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu: > On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote: > > Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu: > > > Add basic clang support in clang.cpp and test__clang() testcase. The > > > first testcase checks if builtin clang is able to generate LLVM IR. > > > > > > tests/clang.c is a proxy. Real testcase resides in > > > utils/c++/clang-test.cpp in c++ and exports C interface to perf test > > > subsystem. > > > > > > Test result: > > > > > > $ perf test -v clang > > > 51: Test builtin clang support : > > > 51.1: Test builtin clang compile C source to IR : > > > --- start --- > > > test child forked, pid 13215 > > > test child finished with 0 > > > ---- end ---- > > > Test builtin clang support subtest 0: Ok > > > > While testing this I noticed that the perf binary got huge, can't this > > be done in some other way, i.e. using dynamic library? > > > > [root@jouet ~]# size /tmp/perf > > text data bss dec hex filename > > 75333688 1421584 23962176 100717448 600d388 /tmp/perf > > [root@jouet ~]# > > > > I've built it with this: > > > > make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin > > > > the resulting binary: > > > > [acme@jouet linux]$ > > [acme@jouet linux]$ ls -la ~/bin/perf > > -rwxr-xr-x. 2 acme acme 131689136 Dec 2 12:31 /home/acme/bin/perf > > [acme@jouet linux]$ ls -lah ~/bin/perf > > -rwxr-xr-x. 2 acme acme 126M Dec 2 12:31 /home/acme/bin/perf > > [acme@jouet linux]$ > > > > Huge, after stripping it: > > > > [acme@jouet linux]$ ls -la /tmp/perf > > -rwxr-xr-x. 1 root root 76759056 Dec 2 12:33 /tmp/perf > > [acme@jouet linux]$ ls -lah /tmp/perf > > -rwxr-xr-x. 1 root root 74M Dec 2 12:33 /tmp/perf > > [acme@jouet linux]$ > > > > Still huge :-\ > > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode. > Try llvm-config --build-mode --assertion-mode > it should be Release OFF Probably this was with 3.9 and built from git, quite a while ago, now I removed it from /usr/local/ and installed what is in f25, but I fear it will be insufficient, does 3.8 cuts it for what we're testing? Humm, it looks like it will: [root@jouet ~]# llc --version LLVM (http://llvm.org/): LLVM version 3.8.0 Optimized build. Built Mar 10 2016 (01:41:45). Default target: x86_64-unknown-linux-gnu Host CPU: broadwell Registered Targets: aarch64 - AArch64 (little endian) aarch64_be - AArch64 (big endian) amdgcn - AMD GCN GPUs arm - ARM arm64 - ARM64 (little endian) armeb - ARM (big endian) bpf - BPF (host endian) bpfeb - BPF (big endian) bpfel - BPF (little endian) cpp - C++ backend nvptx - NVIDIA PTX 32-bit nvptx64 - NVIDIA PTX 64-bit ppc32 - PowerPC 32 ppc64 - PowerPC 64 ppc64le - PowerPC 64 LE r600 - AMD GPUs HD2XXX-HD6XXX systemz - SystemZ thumb - Thumb thumbeb - Thumb (big endian) x86 - 32-bit X86: Pentium-Pro and above x86-64 - 64-bit X86: EM64T and AMD64 [root@jouet ~]# But I'm now running the container based tests to send a pull req, will check later, after that. - Arnaldo > > But being a n00b on llvm/clang libraries, etc, my question goes back to: > > can we have this using a libllvm.so or libclang.so dynamic libraries? > > that can also work. The reason we build iovisor/bcc into single binary > is to ease operational headache.
Em Mon, Dec 05, 2016 at 07:02:48PM -0200, Arnaldo Carvalho de Melo escreveu: > Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu: > > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode. > > Try llvm-config --build-mode --assertion-mode > > it should be Release OFF > > Probably this was with 3.9 and built from git, quite a while ago, now I > removed it from /usr/local/ and installed what is in f25, but I fear it > will be insufficient, does 3.8 cuts it for what we're testing? Humm, it > looks like it will: > > [root@jouet ~]# llc --version > LLVM (http://llvm.org/): > LLVM version 3.8.0 > > But I'm now running the container based tests to send a pull req, will > check later, after that. Not really, Wang, we need to update that feature detection test to state what is the minimum required LLVM/clang version, one that has those functions, which, unfortunately, isn't the one in the latest fedora, fedora 25: [acme@jouet linux]$ rpm -q llvm-devel clang-devel llvm-devel-3.8.0-1.fc25.x86_64 clang-devel-3.8.0-2.fc25.x86_64 [acme@jouet linux]$ LINK /tmp/build/perf/libperf-gtk.so INSTALL GTK UI /tmp/build/perf/libperf.a(libperf-in.o): In function `perf::createCompilerInvocation(llvm::SmallVector<char const*, 16u>, llvm::StringRef&, clang::DiagnosticsEngine&)': /home/acme/git/linux/tools/perf/util/c++/clang.cpp:56: undefined reference to `clang::tooling::newInvocation(clang::DiagnosticsEngine*, llvm::SmallVector<char const*, 16u> const&)' /tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)': /home/acme/git/linux/tools/perf/util/c++/clang.cpp:68: undefined reference to `clang::CompilerInstance::CompilerInstance(std::shared_ptr<clang::PCHContainerOperations>, bool)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:69: undefined reference to `clang::CompilerInstance::createDiagnostics(clang::DiagnosticConsumer*, bool)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:76: undefined reference to `clang::CompilerInstance::setInvocation(clang::CompilerInvocation*)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:78: undefined reference to `clang::EmitLLVMOnlyAction::EmitLLVMOnlyAction(llvm::LLVMContext*)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:79: undefined reference to `clang::CompilerInstance::ExecuteAction(clang::FrontendAction&)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:82: undefined reference to `clang::CodeGenAction::takeModule()' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:68: undefined reference to `clang::CompilerInstance::~CompilerInstance()' /tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef, llvm::StringRef)': /home/acme/git/linux/tools/perf/util/c++/clang.cpp:92: undefined reference to `clang::vfs::getRealFileSystem()' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:92: undefined reference to `clang::vfs::OverlayFileSystem::OverlayFileSystem(llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:94: undefined reference to `clang::vfs::InMemoryFileSystem::InMemoryFileSystem(bool)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:100: undefined reference to `clang::vfs::OverlayFileSystem::pushOverlay(llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)' /home/acme/git/linux/tools/perf/util/c++/clang.cpp:101: undefined reference to `clang::vfs::InMemoryFileSystem::addFile(llvm::Twine const&, long, std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> >)' /tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef)': /home/acme/git/linux/tools/perf/util/c++/clang.cpp:109: undefined reference to `clang::vfs::getRealFileSystem()' /tmp/build/perf/libperf.a(libperf-in.o): In function `clang::CompilerInvocation::~CompilerInvocation()': /usr/include/clang/Frontend/CompilerInvocation.h:102: undefined reference to `clang::CompilerInvocationBase::~CompilerInvocationBase()' /tmp/build/perf/libperf.a(libperf-in.o): In function `void __gnu_cxx::new_allocator<clang::PCHContainerOperations>::construct<clang::PCHContainerOperations>(clang::PCHContainerOperations*)': /usr/include/c++/6.2.1/ext/new_allocator.h:120: undefined reference to `clang::PCHContainerOperations::PCHContainerOperations()' collect2: error: ld returned 1 exit status Makefile.perf:491: recipe for target '/tmp/build/perf/perf' failed make[1]: *** [/tmp/build/perf/perf] Error 1 Makefile:108: recipe for target 'install-bin' failed make: *** [install-bin] Error 2 make: Leaving directory '/home/acme/git/linux/tools/perf' [acme@jouet linux]$
On 2016/12/6 5:48, Arnaldo Carvalho de Melo wrote: > Em Mon, Dec 05, 2016 at 07:02:48PM -0200, Arnaldo Carvalho de Melo escreveu: >> Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu: >>> yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode. >>> Try llvm-config --build-mode --assertion-mode >>> it should be Release OFF >> Probably this was with 3.9 and built from git, quite a while ago, now I >> removed it from /usr/local/ and installed what is in f25, but I fear it >> will be insufficient, does 3.8 cuts it for what we're testing? Humm, it >> looks like it will: >> >> [root@jouet ~]# llc --version >> LLVM (http://llvm.org/): >> LLVM version 3.8.0 >> >> But I'm now running the container based tests to send a pull req, will >> check later, after that. > Not really, Wang, we need to update that feature detection test to state what > is the minimum required LLVM/clang version, one that has those functions, > which, unfortunately, isn't the one in the latest fedora, fedora 25: I'll set the minimum required LLVM version to 3.9, and report warning when LLVM is too old. However, since LLVM interface is keep changing, finally we will have problem if we want to support 2 or 3 different clang/LLVM. We should keep moving minimum requirement LLVM version if we don't want to see '#ifdef's spread in our code. Thank you.
Em Tue, Dec 06, 2016 at 10:07:51AM +0800, Wangnan (F) escreveu: > On 2016/12/6 5:48, Arnaldo Carvalho de Melo wrote: > > Em Mon, Dec 05, 2016 at 07:02:48PM -0200, Arnaldo Carvalho de Melo escreveu: > > > Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu: > > > > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode. > > > > Try llvm-config --build-mode --assertion-mode > > > > it should be Release OFF > > > Probably this was with 3.9 and built from git, quite a while ago, now I > > > removed it from /usr/local/ and installed what is in f25, but I fear it > > > will be insufficient, does 3.8 cuts it for what we're testing? Humm, it > > > looks like it will: > > > [root@jouet ~]# llc --version > > > LLVM (http://llvm.org/): > > > LLVM version 3.8.0 > > > But I'm now running the container based tests to send a pull req, will > > > check later, after that. > > Not really, Wang, we need to update that feature detection test to state what > > is the minimum required LLVM/clang version, one that has those functions, > > which, unfortunately, isn't the one in the latest fedora, fedora 25: > I'll set the minimum required LLVM version to 3.9, and report > warning when LLVM is too old. However, since LLVM interface is > keep changing, finally we will have problem if we want to support > 2 or 3 different clang/LLVM. We should keep moving minimum > requirement LLVM version if we don't want to see '#ifdef's > spread in our code. If this area is in that much flux, I see no problem in us both not enabling this by default, which is the case right now, and go on moving the minimum required version for LLVM/clang, hopefully at some point this will get stable and widely available (as in what distros ship), when we then switch to doing the feature detection automatically. I see you already submitted the patch to do this test, thanks, will check and continue processing your series. - Arnaldo
Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu: > On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote: > > Em Sat, Nov 26, 2016 at 07:03:34AM +0000, Wang Nan escreveu: > > > Add basic clang support in clang.cpp and test__clang() testcase. The > > > first testcase checks if builtin clang is able to generate LLVM IR. > > > > > > tests/clang.c is a proxy. Real testcase resides in > > > utils/c++/clang-test.cpp in c++ and exports C interface to perf test > > > subsystem. > > > > > > Test result: > > > > > > $ perf test -v clang > > > 51: Test builtin clang support : > > > 51.1: Test builtin clang compile C source to IR : > > > --- start --- > > > test child forked, pid 13215 > > > test child finished with 0 > > > ---- end ---- > > > Test builtin clang support subtest 0: Ok > > > > While testing this I noticed that the perf binary got huge, can't this > > be done in some other way, i.e. using dynamic library? > > > > [root@jouet ~]# size /tmp/perf > > text data bss dec hex filename > > 75333688 1421584 23962176 100717448 600d388 /tmp/perf > > [root@jouet ~]# > > > > I've built it with this: > > > > make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin > > > > the resulting binary: > > > > [acme@jouet linux]$ > > [acme@jouet linux]$ ls -la ~/bin/perf > > -rwxr-xr-x. 2 acme acme 131689136 Dec 2 12:31 /home/acme/bin/perf > > [acme@jouet linux]$ ls -lah ~/bin/perf > > -rwxr-xr-x. 2 acme acme 126M Dec 2 12:31 /home/acme/bin/perf > > [acme@jouet linux]$ > > > > Huge, after stripping it: > > > > [acme@jouet linux]$ ls -la /tmp/perf > > -rwxr-xr-x. 1 root root 76759056 Dec 2 12:33 /tmp/perf > > [acme@jouet linux]$ ls -lah /tmp/perf > > -rwxr-xr-x. 1 root root 74M Dec 2 12:33 /tmp/perf > > [acme@jouet linux]$ > > > > Still huge :-\ > > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode. > Try llvm-config --build-mode --assertion-mode > it should be Release OFF It was ON, rebuilding it with $ cmake -DCMAKE_BUILD_TYPE=Release /home/acme/git/llvm $ make -j4 Will take a (another) while :-) - Arnaldo > > But being a n00b on llvm/clang libraries, etc, my question goes back to: > > can we have this using a libllvm.so or libclang.so dynamic libraries? > > that can also work. The reason we build iovisor/bcc into single binary > is to ease operational headache.
Em Tue, Dec 06, 2016 at 04:19:34PM -0300, Arnaldo Carvalho de Melo escreveu: > Em Mon, Dec 05, 2016 at 08:51:01AM -0800, Alexei Starovoitov escreveu: > > On Fri, Dec 02, 2016 at 01:44:40PM -0200, Arnaldo Carvalho de Melo wrote: > > > make LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin > > > > > > the resulting binary: > > > > > > [acme@jouet linux]$ > > > [acme@jouet linux]$ ls -la ~/bin/perf > > > -rwxr-xr-x. 2 acme acme 131689136 Dec 2 12:31 /home/acme/bin/perf > > > [acme@jouet linux]$ ls -lah ~/bin/perf > > > -rwxr-xr-x. 2 acme acme 126M Dec 2 12:31 /home/acme/bin/perf > > > [acme@jouet linux]$ > > > > > > Huge, after stripping it: > > > > > > [acme@jouet linux]$ ls -la /tmp/perf > > > -rwxr-xr-x. 1 root root 76759056 Dec 2 12:33 /tmp/perf > > > [acme@jouet linux]$ ls -lah /tmp/perf > > > -rwxr-xr-x. 1 root root 74M Dec 2 12:33 /tmp/perf > > > [acme@jouet linux]$ > > > > > > Still huge :-\ > > > > yeah. it's kinda high. I'm guessing rpm llvm libs are in debug mode. > > Try llvm-config --build-mode --assertion-mode > > it should be Release OFF > > It was ON, rebuilding it with > > $ cmake -DCMAKE_BUILD_TYPE=Release /home/acme/git/llvm > $ make -j4 > > Will take a (another) while :-) Almost halved the size: [acme@jouet build]$ llvm-config --build-mode --assertion-mode Release OFF [acme@jouet build]$ clang -v clang version 4.0.0 (http://llvm.org/git/clang.git b7a0d79a6691813bf7d6ade1b4e8b21fd502f50a) (http://llvm.org/git/llvm.git 3a783f8716bfc621355e8ae61daf3a2093c341fc) Target: x86_64-unknown-linux-gnu Thread model: posix InstalledDir: /usr/local/bin Found candidate GCC installation: /usr/lib/gcc/x86_64-redhat-linux/6.2.1 Selected GCC installation: /usr/lib/gcc/x86_64-redhat-linux/6.2.1 Candidate multilib: .;@m64 Candidate multilib: 32;@m32 Selected multilib: .;@m64 [acme@jouet build]$ llvm-config --version 4.0.0svn [acme@jouet build]$ [acme@jouet linux]$ ls -la /tmp/perf -rwxr-xr-x. 1 acme acme 63641016 Dec 6 17:58 /tmp/perf [acme@jouet linux]$ size /tmp/perf text data bss dec hex filename 40877921 1577232 24065712 66520865 3f70721 /tmp/perf [acme@jouet linux]$ strip /tmp/perf [acme@jouet linux]$ ls -la /tmp/perf -rwxr-xr-x. 1 acme acme 42460904 Dec 6 17:58 /tmp/perf [acme@jouet linux]$ ls -lah /tmp/perf -rwxr-xr-x. 1 acme acme 41M Dec 6 17:58 /tmp/perf [acme@jouet linux]$ size /tmp/perf text data bss dec hex filename 40877921 1577232 24065712 66520865 3f70721 /tmp/perf [acme@jouet linux]$
diff --git a/tools/perf/tests/Build b/tools/perf/tests/Build index af3ec94..6676c2d 100644 --- a/tools/perf/tests/Build +++ b/tools/perf/tests/Build @@ -43,6 +43,7 @@ perf-y += sdt.o perf-y += is_printable_array.o perf-y += bitmap.o perf-y += perf-hooks.o +perf-y += clang.o $(OUTPUT)tests/llvm-src-base.c: tests/bpf-script-example.c tests/Build $(call rule_mkdir) diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c index dab83f7..33aaa52 100644 --- a/tools/perf/tests/builtin-test.c +++ b/tools/perf/tests/builtin-test.c @@ -234,6 +234,15 @@ static struct test generic_tests[] = { .func = test__perf_hooks, }, { + .desc = "Test builtin clang support", + .func = test__clang, + .subtest = { + .skip_if_fail = true, + .get_nr = test__clang_subtest_get_nr, + .get_desc = test__clang_subtest_get_desc, + } + }, + { .func = NULL, }, }; diff --git a/tools/perf/tests/clang.c b/tools/perf/tests/clang.c new file mode 100644 index 0000000..57ee160 --- /dev/null +++ b/tools/perf/tests/clang.c @@ -0,0 +1,42 @@ +#include "tests.h" +#include "debug.h" +#include "util.h" +#include "c++/clang-c.h" + +static struct { + int (*func)(void); + const char *desc; +} clang_testcase_table[] = { +#ifdef HAVE_LIBCLANGLLVM_SUPPORT + { + .func = test__clang_to_IR, + .desc = "Test builtin clang compile C source to IR", + }, +#endif +}; + +int test__clang_subtest_get_nr(void) +{ + return (int)ARRAY_SIZE(clang_testcase_table); +} + +const char *test__clang_subtest_get_desc(int i) +{ + if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table)) + return NULL; + return clang_testcase_table[i].desc; +} + +#ifndef HAVE_LIBCLANGLLVM_SUPPORT +int test__clang(int i __maybe_unused) +{ + return TEST_SKIP; +} +#else +int test__clang(int i __maybe_unused) +{ + if (i < 0 || i >= (int)ARRAY_SIZE(clang_testcase_table)) + return TEST_FAIL; + return clang_testcase_table[i].func(); +} +#endif diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h index 3a1f98f..0d7b251 100644 --- a/tools/perf/tests/tests.h +++ b/tools/perf/tests/tests.h @@ -92,6 +92,9 @@ int test__sdt_event(int subtest); int test__is_printable_array(int subtest); int test__bitmap_print(int subtest); int test__perf_hooks(int subtest); +int test__clang(int subtest); +const char *test__clang_subtest_get_desc(int subtest); +int test__clang_subtest_get_nr(void); #if defined(__arm__) || defined(__aarch64__) #ifdef HAVE_DWARF_UNWIND_SUPPORT diff --git a/tools/perf/util/Build b/tools/perf/util/Build index b2a47aa..743a889 100644 --- a/tools/perf/util/Build +++ b/tools/perf/util/Build @@ -125,6 +125,8 @@ endif libperf-y += perf-hooks.o +libperf-$(CONFIG_CXX) += c++/ + CFLAGS_config.o += -DETC_PERFCONFIG="BUILD_STR($(ETC_PERFCONFIG_SQ))" # avoid compiler warnings in 32-bit mode CFLAGS_genelf_debug.o += -Wno-packed diff --git a/tools/perf/util/c++/Build b/tools/perf/util/c++/Build new file mode 100644 index 0000000..988fef1 --- /dev/null +++ b/tools/perf/util/c++/Build @@ -0,0 +1,2 @@ +libperf-$(CONFIG_CLANGLLVM) += clang.o +libperf-$(CONFIG_CLANGLLVM) += clang-test.o diff --git a/tools/perf/util/c++/clang-c.h b/tools/perf/util/c++/clang-c.h new file mode 100644 index 0000000..dcde4b5 --- /dev/null +++ b/tools/perf/util/c++/clang-c.h @@ -0,0 +1,16 @@ +#ifndef PERF_UTIL_CLANG_C_H +#define PERF_UTIL_CLANG_C_H + +#ifdef __cplusplus +extern "C" { +#endif + +extern void perf_clang__init(void); +extern void perf_clang__cleanup(void); + +extern int test__clang_to_IR(void); + +#ifdef __cplusplus +} +#endif +#endif diff --git a/tools/perf/util/c++/clang-test.cpp b/tools/perf/util/c++/clang-test.cpp new file mode 100644 index 0000000..3da6bfa --- /dev/null +++ b/tools/perf/util/c++/clang-test.cpp @@ -0,0 +1,31 @@ +#include "clang.h" +#include "clang-c.h" +#include "llvm/IR/Function.h" +#include "llvm/IR/LLVMContext.h" + +class perf_clang_scope { +public: + explicit perf_clang_scope() {perf_clang__init();} + ~perf_clang_scope() {perf_clang__cleanup();} +}; + +extern "C" { + +int test__clang_to_IR(void) +{ + perf_clang_scope _scope; + + std::unique_ptr<llvm::Module> M = + perf::getModuleFromSource("perf-test.c", + "int myfunc(void) {return 1;}"); + + if (!M) + return -1; + + for (llvm::Function& F : *M) + if (F.getName() == "myfunc") + return 0; + return -1; +} + +} diff --git a/tools/perf/util/c++/clang.cpp b/tools/perf/util/c++/clang.cpp new file mode 100644 index 0000000..c17b117 --- /dev/null +++ b/tools/perf/util/c++/clang.cpp @@ -0,0 +1,96 @@ +/* + * llvm C frontend for perf. Support dynamically compile C file + * + * Inspired by clang example code: + * http://llvm.org/svn/llvm-project/cfe/trunk/examples/clang-interpreter/main.cpp + * + * Copyright (C) 2016 Wang Nan <wangnan0@huawei.com> + * Copyright (C) 2016 Huawei Inc. + */ + +#include "clang/CodeGen/CodeGenAction.h" +#include "clang/Frontend/CompilerInvocation.h" +#include "clang/Frontend/CompilerInstance.h" +#include "clang/Frontend/TextDiagnosticPrinter.h" +#include "clang/Tooling/Tooling.h" +#include "llvm/IR/Module.h" +#include "llvm/Option/Option.h" +#include "llvm/Support/ManagedStatic.h" +#include <memory> + +#include "clang.h" +#include "clang-c.h" + +namespace perf { + +static std::unique_ptr<llvm::LLVMContext> LLVMCtx; + +using namespace clang; + +static vfs::InMemoryFileSystem * +buildVFS(StringRef& Name, StringRef& Content) +{ + vfs::InMemoryFileSystem *VFS = new vfs::InMemoryFileSystem(true); + VFS->addFile(Twine(Name), 0, llvm::MemoryBuffer::getMemBuffer(Content)); + return VFS; +} + +static CompilerInvocation * +createCompilerInvocation(StringRef& Path, DiagnosticsEngine& Diags) +{ + llvm::opt::ArgStringList CCArgs { + "-cc1", + "-triple", "bpf-pc-linux", + "-fsyntax-only", + "-ferror-limit", "19", + "-fmessage-length", "127", + "-O2", + "-nostdsysteminc", + "-nobuiltininc", + "-vectorize-loops", + "-vectorize-slp", + "-Wno-unused-value", + "-Wno-pointer-sign", + "-x", "c"}; + CompilerInvocation *CI = tooling::newInvocation(&Diags, CCArgs); + + FrontendOptions& Opts = CI->getFrontendOpts(); + Opts.Inputs.clear(); + Opts.Inputs.emplace_back(Path, IK_C); + return CI; +} + +std::unique_ptr<llvm::Module> +getModuleFromSource(StringRef Name, StringRef Content) +{ + CompilerInstance Clang; + Clang.createDiagnostics(); + + IntrusiveRefCntPtr<vfs::FileSystem> VFS = buildVFS(Name, Content); + Clang.setVirtualFileSystem(&*VFS); + + IntrusiveRefCntPtr<CompilerInvocation> CI = + createCompilerInvocation(Name, Clang.getDiagnostics()); + Clang.setInvocation(&*CI); + + std::unique_ptr<CodeGenAction> Act(new EmitLLVMOnlyAction(&*LLVMCtx)); + if (!Clang.ExecuteAction(*Act)) + return std::unique_ptr<llvm::Module>(nullptr); + + return Act->takeModule(); +} + +} + +extern "C" { +void perf_clang__init(void) +{ + perf::LLVMCtx.reset(new llvm::LLVMContext()); +} + +void perf_clang__cleanup(void) +{ + perf::LLVMCtx.reset(nullptr); + llvm::llvm_shutdown(); +} +} diff --git a/tools/perf/util/c++/clang.h b/tools/perf/util/c++/clang.h new file mode 100644 index 0000000..f64483b --- /dev/null +++ b/tools/perf/util/c++/clang.h @@ -0,0 +1,16 @@ +#ifndef PERF_UTIL_CLANG_H +#define PERF_UTIL_CLANG_H + +#include "llvm/ADT/StringRef.h" +#include "llvm/IR/LLVMContext.h" +#include "llvm/IR/Module.h" +#include <memory> +namespace perf { + +using namespace llvm; + +std::unique_ptr<Module> +getModuleFromSource(StringRef Name, StringRef Content); + +} +#endif
Add basic clang support in clang.cpp and test__clang() testcase. The first testcase checks if builtin clang is able to generate LLVM IR. tests/clang.c is a proxy. Real testcase resides in utils/c++/clang-test.cpp in c++ and exports C interface to perf test subsystem. Test result: $ perf test -v clang 51: Test builtin clang support : 51.1: Test builtin clang compile C source to IR : --- start --- test child forked, pid 13215 test child finished with 0 ---- end ---- Test builtin clang support subtest 0: Ok Signed-off-by: Wang Nan <wangnan0@huawei.com> Cc: Arnaldo Carvalho de Melo <acme@redhat.com> Cc: Alexei Starovoitov <ast@fb.com> Cc: He Kuang <hekuang@huawei.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Zefan Li <lizefan@huawei.com> Cc: pi3orama@163.com --- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 9 ++++ tools/perf/tests/clang.c | 42 +++++++++++++++++ tools/perf/tests/tests.h | 3 ++ tools/perf/util/Build | 2 + tools/perf/util/c++/Build | 2 + tools/perf/util/c++/clang-c.h | 16 +++++++ tools/perf/util/c++/clang-test.cpp | 31 ++++++++++++ tools/perf/util/c++/clang.cpp | 96 ++++++++++++++++++++++++++++++++++++++ tools/perf/util/c++/clang.h | 16 +++++++ 10 files changed, 218 insertions(+) create mode 100644 tools/perf/tests/clang.c create mode 100644 tools/perf/util/c++/Build create mode 100644 tools/perf/util/c++/clang-c.h create mode 100644 tools/perf/util/c++/clang-test.cpp create mode 100644 tools/perf/util/c++/clang.cpp create mode 100644 tools/perf/util/c++/clang.h -- 2.10.1