提问者:小点点

Apple M1的Tensorflow无法按预期工作。尝试训练我的Keras模型时总是得到PIL错误


我昨天买了一台带有M1芯片的Mac mini,因为我的2018款Mac mini在训练NN时总是非常热,而且需要很长时间。现在,我在这款Mac上安装了Tensorflow,一次是在venv中安装的,一次在venv上使用Rosetta进行了仿真。rosetta版本的工作与预期的一样(但比我的旧Mac mini 2018和i5慢?)但我无法让本机版本的代码正确运行,因为我总是收到下面的错误
我也无法在本地安装Matplotlib、sklearn和许多其他软件包。

代码(来自deeplizard的课程:

import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Activation, Dense, Flatten, BatchNormalization, Conv2D, MaxPool2D
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.metrics import categorical_crossentropy
from tensorflow.keras.preprocessing.image import ImageDataGenerator
import itertools
import os
import shutil
import random
import glob
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

os.chdir('data/dogs-vs-cats')
if os.path.isdir('train/dog') is False:
    os.makedirs('train/dog')
    os.makedirs('train/cat')
    os.makedirs('valid/dog')
    os.makedirs('valid/cat')
    os.makedirs('test/dog')
    os.makedirs('test/cat')

    for i in random.sample(glob.glob('cat*'), 500):
        shutil.move(i, 'train/cat')
    for i in random.sample(glob.glob('dog*'), 500):
        shutil.move(i, 'train/dog')
    for i in random.sample(glob.glob('cat*'), 100):
        shutil.move(i, 'valid/cat')
    for i in random.sample(glob.glob('dog*'), 100):
        shutil.move(i, 'valid/dog')
    for i in random.sample(glob.glob('cat*'), 50):
        shutil.move(i, 'test/cat')
    for i in random.sample(glob.glob('dog*'), 50):
        shutil.move(i, 'test/dog')

os.chdir('../../')

train_path = 'data/dogs-vs-cats/train'
valid_path = 'data/dogs-vs-cats/valid'
test_path = 'data/dogs-vs-cats/test'

train_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input) \
    .flow_from_directory(directory=train_path, target_size=(224,224), classes=['cat', 'dog'], batch_size=10)
valid_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input) \
    .flow_from_directory(directory=valid_path, target_size=(224,224), classes=['cat', 'dog'], batch_size=10)
test_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input) \
    .flow_from_directory(directory=test_path, target_size=(224,224), classes=['cat', 'dog'], batch_size=10, shuffle=False)

assert train_batches.n == 1000
assert valid_batches.n == 200
assert test_batches.n == 100

imgs, labels = next(train_batches)
print(labels)



vgg16_model = tf.keras.applications.vgg16.VGG16()
vgg16_model.summary()

model = Sequential()
for layer in vgg16_model.layers[:-1]:   #loop through vgg16 and copy the layers to new model, except the last one!
    model.add(layer)

model.summary()

for layer in model.layers:  #iterate over all new layers in sequential model and set them to be not trainable
    layer.trainable = False

model.add(Dense(units=2, activation='softmax'))

model.summary()

model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(x=train_batches, validation_data=valid_batches, epochs=5, verbose=2)

终端输出:

Found 1000 images belonging to 2 classes.
Found 200 images belonging to 2 classes.
Found 100 images belonging to 2 classes.
Traceback (most recent call last):
  File "main.py", line 56, in <module>
    imgs, labels = next(train_batches)
  File "/Users/florianraab/tensorflow_macos_venv/lib/python3.8/site-packages/keras_preprocessing/image/iterator.py", line 104, in __next__
    return self.next(*args, **kwargs)
  File "/Users/florianraab/tensorflow_macos_venv/lib/python3.8/site-packages/keras_preprocessing/image/iterator.py", line 116, in next
    return self._get_batches_of_transformed_samples(index_array)
  File "/Users/florianraab/tensorflow_macos_venv/lib/python3.8/site-packages/keras_preprocessing/image/iterator.py", line 227, in _get_batches_of_transformed_samples
    img = load_img(filepaths[j],
  File "/Users/florianraab/tensorflow_macos_venv/lib/python3.8/site-packages/keras_preprocessing/image/utils.py", line 111, in load_img
    raise ImportError('Could not import PIL.Image. '
ImportError: Could not import PIL.Image. The use of `load_img` requires PIL.

尝试安装枕头后

pip3 install pillow

我得到以下输出:

Requirement already satisfied: pillow in /Users/florianraab/tensorflow_macos_venv/lib/python3.8/site-packages (8.0.1)

任何人都可以指导我解决这个问题吗?

否则我想我会退回 M1 并继续使用我的 2018 Mac mini,尽管我总是担心,因为训练时恒定的 100°C


共1个答案

匿名用户

我希望你已经找到了答案,但如果没有,这里有一些建议可能会有所帮助:

>

  • 针对 M1 优化的 TensorFlow 设置起来并不难,但边缘仍然有点粗糙。也许稍后返回并重新获得 M1 是个好主意,除非您愿意在某些深度学习冒险中忍受类似开发人员预览的环境。

    如果您查看导致最低级别问题的实际utils.py文件,您会注意到该异常是由以下导入中的一个失败引发的:

    from PIL import ImageEnhance
    from PIL import Image as pil_image
    

    确保它们正常工作。如果没有,考虑卸载并重新安装Pillow。确保终端上的Rosetta设置是一致的。这一部分应该很好地与M1合作。

    此后,如果您的例程利用了< code>affine_transform功能(即用于图像放大),您可能会收到一个SciPy错误。如果发生这种情况,运气不好;截至12月20日,SciPy还不能与M1 macbook兼容,因为它需要Fortran编译。

    短期内,如果您确信apply_affine_transform不会对模型的学习过程产生影响,您可以暂时将更改为非ciPy检查标识函数。否则,您可以尝试删除keras.Image.apply_affine_transform对ciPy的依赖,尽管我不推荐这样做(我愿意付出的努力无法做到这一点)。