Lazy IO is bad, and this is generally considered to be a pain point in Haskell. Basically the contents
isn't evaluated until you go to write it back to disk, at which point it can't be evaluated because the file is already closed. You can fix this in several ways, without resorting to extra libraries you can use the readFile
function and then check the length before writing back out:
import Control.Monad (when)
main = do
contents <- readFile "file.txt"
let newContents = map toUpper contents
when (length newContents > 0) $
writeFile "file.txt" newContents
I would say this code is actually better anyway because you don't write back out to a file that is already empty, a pointless operation.
Another way would be to use a streaming library, pipes
is a popular choice with some good tutorials and a solid mathematical foundation, and that would be my choice as well.
Tensorflow compiled with the advanced instructions can run 40% faster than the non-optimised code, so please don't ignore this message!
I would recommend building TF from source, here are the steps to download, compile and install v2.0 beta, including the link to the compiled .whl file.
First, uninstall the default version:
sudo pip3 uninstall protobuf
sudo pip3 uninstall tensorflow
In a temp folder, clone Tensorflow:
git clone https://github.com/tensorflow/tensorflow
git checkout r2.0
Install the TensorFlow pip package dependencies:
pip3 install -U --user pip six numpy wheel setuptools mock future>=0.17.1
pip3 install -U --user keras_applications==1.0.6 --no-deps
pip3 install -U --user keras_preprocessing==1.0.5 --no-deps
Install Bazel, the build tool used to compile TensorFlow. After downloading bazel-0.26.0-installer-darwin-x86_64.sh:
chmod +x bazel-0.26.0-installer-darwin-x86_64.sh ./bazel-0.26.0-installer-darwin-x86_64.sh --user export PATH="$PATH:$HOME/bin" bazel version
Configure your system build by running the following at the root of your TensorFlow source tree:
./configure
Use bazel to make the TensorFlow package builder with CPU-only support:
bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-msse4.2 //tensorflow/tools/pip_package:build_pip_package
The bazel build command creates an executable named build_pip_package—this is the program that builds the pip package. Run the executable as shown below to build a .whl package in the /tmp/tensorflow_pkg directory.
To build from a release branch:
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
The output wheel file is in: /tmp/tensorflow_pkg. You may download the file from here, and try to install it directly
pip3 install /tmp/tensorflow_pkg/tensorflow-2.0.0b1-cp37-cp37m-macosx_10_14_x86_64.whl
cd out of that directory, and now running this should not produce any warning:
python3 -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
The original post was published here.
Best Answer
What is this warning about?
Modern CPUs provide a lot of low-level instructions, besides the usual arithmetic and logic, known as extensions, e.g. SSE2, SSE4, AVX, etc. From the Wikipedia:
In particular, AVX introduces fused multiply-accumulate (FMA) operations, which speed up linear algebra computation, namely dot-product, matrix multiply, convolution, etc. Almost every machine-learning training involves a great deal of these operations, hence will be faster on a CPU that supports AVX and FMA (up to 300%). The warning states that your CPU does support AVX (hooray!).
I'd like to stress here: it's all about CPU only.
Why isn't it used then?
Because tensorflow default distribution is built without CPU extensions, such as SSE4.1, SSE4.2, AVX, AVX2, FMA, etc. The default builds (ones from
pip install tensorflow
) are intended to be compatible with as many CPUs as possible. Another argument is that even with these extensions CPU is a lot slower than a GPU, and it's expected for medium- and large-scale machine-learning training to be performed on a GPU.What should you do?
If you have a GPU, you shouldn't care about AVX support, because most expensive ops will be dispatched on a GPU device (unless explicitly set not to). In this case, you can simply ignore this warning by
... or by setting
export TF_CPP_MIN_LOG_LEVEL=2
if you're on Unix. Tensorflow is working fine anyway, but you won't see these annoying warnings.If you don't have a GPU and want to utilize CPU as much as possible, you should build tensorflow from the source optimized for your CPU with AVX, AVX2, and FMA enabled if your CPU supports them. It's been discussed in this question and also this GitHub issue. Tensorflow uses an ad-hoc build system called bazel and building it is not that trivial, but is certainly doable. After this, not only will the warning disappear, tensorflow performance should also improve.