I am following TensorFlow's Image Segmentation tutorial. In there there are the following lines:
train_dataset = train.cache().shuffle(BUFFER_SIZE).batch(BATCH_SIZE).repeat()
train_dataset = train_dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)
What does the
cache()
function do? The official documentation is pretty obscure and self-referencing:
![TensorFlow 特征列介绍](https://p.xsw88.cn/allimgs/daicuo/20230906/1202.png)
Caches the elements in this dataset.
What does the
prefetch()
function do? The official documentation is again pretty obscure:
Creates a Dataset that prefetches elements from this dataset.
解决方案
The tf.data.Dataset.cache
transformation can cache a dataset, either in memory or on local storage. This will save some operations (like file opening and data reading) from being executed during each epoch. The next epochs will reuse the data cached by the cache transformation.
You can find more about the cache
in tensorflow here.
Prefetch
overlaps the preprocessing and model execution of a training step. While the model is executing training step s, the input pipeline is reading the data for step s+1. Doing so reduces the step time to the maximum (as opposed to the sum) of the training and the time it takes to extract the data.
You can find more about prefetch
in tensorflow here.
Hope this answers your question. Happy Learning.
相关推荐
最新文章