oss-fuzz/infra/base-images/base-sanitizer-libs-builder/compiler_wrapper.py

177 lines
4.0 KiB
Python
Raw Normal View History

#!/usr/bin/env python
# Copyright 2017 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
################################################################################
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
"""Wrapper script to call clang or GCC with modified arguments."""
from __future__ import print_function
import os
import subprocess
import sys
import msan_build
GCC_ONLY_ARGS = [
'-aux-info',
]
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
M32_BIT_ARGS = [
'-m32',
'-mx32',
]
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def invoked_as_gcc():
"""Return whether or not we're pretending to be GCC."""
return sys.argv[0].endswith('gcc') or sys.argv[0].endswith('g++')
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def is_32_bit(args):
"""Return whether or not we're 32-bit."""
return any(arg in M32_BIT_ARGS for arg in args)
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def filter_wl_arg(arg):
"""Remove -z,defs and equivalents from a single -Wl option."""
parts = arg.split(',')[1:]
filtered = []
for part in parts:
2018-01-16 13:26:18 +00:00
if part == 'defs':
removed = filtered.pop()
assert removed == '-z'
continue
2018-01-16 13:26:18 +00:00
if part == '--no-undefined':
continue
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
filtered.append(part)
if filtered:
return '-Wl,' + ','.join(filtered)
# Filtered entire argument.
return None
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def _remove_last_matching(args, find):
for i in range(len(args) - 1, -1, -1):
if args[i] == find:
del args[i]
return
raise IndexError('Not found')
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def remove_z_defs(args):
"""Remove unsupported -Wl,-z,defs linker option."""
filtered = []
for arg in args:
2018-01-16 13:26:18 +00:00
if arg == '-Wl,defs':
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
_remove_last_matching(filtered, '-Wl,-z')
continue
2018-01-16 13:26:18 +00:00
if arg == '-Wl,--no-undefined':
continue
if arg.startswith('-Wl,'):
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
arg = filter_wl_arg(arg)
if not arg:
continue
filtered.append(arg)
return filtered
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def get_compiler_args(args, is_cxx):
"""Generate compiler args."""
compiler_args = args[1:]
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
if is_32_bit(args):
# 32 bit builds not supported.
compiler_args.extend([
'-fno-sanitize=memory',
'-fno-sanitize-memory-track-origins',
])
return compiler_args
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
compiler_args = remove_z_defs(compiler_args)
compiler_args.extend([
# FORTIFY_SOURCE is not supported by sanitizers.
'-U_FORTIFY_SOURCE',
'-Wp,-U_FORTIFY_SOURCE',
# Reduce binary size.
'-gline-tables-only',
# Disable all warnings.
'-w',
2018-02-20 05:41:33 +00:00
# LTO isn't supported.
'-fno-lto',
])
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
if invoked_as_gcc():
compiler_args.extend([
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
# For better compatibility with flags passed via -Wa,...
'-fno-integrated-as',
])
if '-fsanitize=memory' not in args:
# If MSan flags weren't added for some reason, add them here.
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
compiler_args.extend(msan_build.get_injected_flags())
2018-01-18 03:35:01 +00:00
if is_cxx:
compiler_args.append('-stdlib=libc++')
return compiler_args
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def find_real_clang():
"""Return path to real clang."""
return os.environ['REAL_CLANG_PATH']
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
def fallback_to_gcc(args):
"""Check whether if we should fall back to GCC."""
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
if not invoked_as_gcc():
return False
return any(arg in GCC_ONLY_ARGS for arg in args[1:])
def main(args):
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
"""Modify arguments and call the real compiler."""
if fallback_to_gcc(args):
sys.exit(
subprocess.call(['/usr/bin/' + os.path.basename(args[0])] + args[1:]))
is_cxx = args[0].endswith('++')
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
real_clang = find_real_clang()
if is_cxx:
real_clang += '++'
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
args = [real_clang] + get_compiler_args(args, is_cxx)
debug_log_path = os.getenv('WRAPPER_DEBUG_LOG_PATH')
if debug_log_path:
Clean up MemorySanitizer library warnings (#4694) * Update infra/base-images/all.sh Add build of base-sanitizer-libs-builder and msan-libs-builder to this shell script. * msan: Don't warn on un-instrumented standard libs These libraries do not need to be built with instrumentation, because MemorySanitizer includes interceptors for them. * Fix indentation * Add missing docstrings * Fix unused variable * Fix invalid names * Install python-apt on CI * Revert "Install python-apt on CI" This reverts commit d3da49cf90f138674c1a8bcbdca2f0dfe6c385d1. * Install and use python-apt in system directory * Revert "Install and use python-apt in system directory" This reverts commit e0ede101fbdfbf82da5f32d0a206da247e6dde98. * Build python-apt from source * Check out correct version of python-apt * Fix octal literals * More indentation fixes * Add more missing docstrings * Change variable names of opened files * Remove unused import * Ignore lints about package.Package API * Fix or ignore remaining invalid names * Fix apparent typo in compiler_wrapper_test.py -z should precede a keyword, not a long option * Fix use of xrange * Style fixes, compiler_wrapper * Fix apparent error in compiler_wrapper_test.py Similar to the previous error, the test case would pass "-z --no-undefined" to the linker. "-z" only has an effect when it is followed by a keyword, otherwise ld ignores it and prints a warning message. In this test case, "-z" and "--no-undefined" were passed in two separate "-Wl," compiler arguments, but they reflect a common issue. * Add missing license header * Rename more functions * Better name for global variable * Rename methods of Package * Rename functions in msan_builder.py * Fix invalid variable names * Fix useless-object-inheritance * pylint: Fixes for Package and its subclasses * Remove unused imports * Indentation fixes * Fix too-may-locals error in msan_build.py * Add missing docstrings
2020-12-02 20:52:36 +00:00
with open(debug_log_path, 'a') as log_file:
log_file.write(str(args) + '\n')
sys.exit(subprocess.call(args))
if __name__ == '__main__':
main(sys.argv)