1. Introduction

Figure 1. JPEG compression quantization for luminance components. 1) Creation of the quantization table ${\mathit{T}}_Q$, 2) The quantization process, 3) The dequantization process.

Figure 1. JPEG compression quantization for luminance components. 1) Creation of the quantization table ${\mathit{T}}_Q$, 2) The quantization process, 3) The dequantization process.

2. Preliminary: JPEG Quantization

3. Related Works

3.1. ReDMark

Figure 2 shows the overall structure of ReDMark [11], which consists of an embedding network, an extraction network, and an attack layer. The h and w are the height and width of the 2D watermark, and H and W are the height and width of the original and stego-images. The images are divided into $M\times N$-pixel blocks, where $M=N=8$ as it is in ReDMark.

The process flow during training is illustrated by the red arrows in Figure 2. The host image and watermark are fed to the embedding network, and the attack layer degrades the generated stego-image. By feeding the degraded image to the extraction network, the extraction network learns to extract the watermark from the degraded image. After training, the embedding and extraction networks are used individually. The process flow during testing is illustrated by the blue arrows in Figure 2. A stego-image is generated by the embedding network. This image is published and attacked. Let us assume that the attack is to compress the image by some JPEG tool. When the attacked image is obtained, the watermark is extracted from the image in the extraction network.

In ReDMark [11], normalization and reshaping are performed on the input image in preprocessing. For the input image $I_{\mathrm{in}}(i,j),i=0,1,\cdots ,H-1,j=0,1,\cdots ,W-1$, the normalized image is given by

$$I(i,j)=\frac{I_{\mathrm{in}}(i,j)-128}{255}.$$

(7)

Figure 3. Embedding network of ReDMark [11]

Figure 3. Embedding network of ReDMark [11]

Reshape is an operation that divides an image into $M\times N$-pixel blocks and transforms them into a 3D tensor representation. The image size $H\times W$ is assumed to satisfy $H=hM,W=wN$. The reshaped image is represented by a 3-dimensional tensor of $h\times w\times MN$. This image is called the image tensor of size $(h,w,MN)$. If necessary, the tensor is inverse transformed back to its original dimension.

3.1.1. Embedding Network

The embedding network, as shown in Figure 3, consists of three layers: convolution, circular convolution, and transform. The transform layer can perform lossless linear transforms using $1\times 1$ convolutional layers, e.g., the DCT, wavelet transform, and Hadamard transform. In our method, the DCT is selected as the transform layer as it is in ReDMark. The circular convolution layer extends the input to make it cyclic before the convolution is performed. Figure 4 shows an example of applying a circular convolution layer with a $2\times 2$ filter when the input is $3\times 3$ pixels. When a circular convolution layer is used, the dimension of the output after convolution is the same as the dimension of the input.

In the embedding network, the convolution and circular convolution layers use $1\times 1$ and $2\times 2$ filters, respectively, and both have 64 filters. In each layer, an exponential linear unit (ELU) [17] activation function is used. The output of the embedding network is obtained by summing the output of the transform layer performing the inverse DCT (IDCT) (for the IDCT layer) with the input image tensor of size $(h,w,MN)$ and then by performing the inverse process of reshaping. Here, the output of the IDCT layer can be adjusted by the embedding intensity $\alpha$. The intensity is fixed as $\alpha =1$ during training and can be changed during an evaluation.

3.1.2. Extraction network

The extraction network consists of a convolution layer, a circular convolution layer, and a transform layer, as shown in Figure 5. The transform layer of the extraction network also performs the DCT. The filter sizes of the convolutional and circular convolutional layers are $1\times 1$ and $2\times 2$, respectively. The number of filters is 64 for the fourth layer and 1 for the fifth layer. The activation function up to the fourth layer is ELU [17], and a sigmoid function is used in the fifth layer. Let $p_o(i,j)$ be the output of the extraction network, and the estimated watermark $p_e(i,j)$ is given by

$$p_e(i,j)=\left\{\begin{array}{cc}1,\hfill & p_o(i,j)>0.5\hfill \\ 0,\hfill & p_o(i,j)\le 0.5\hfill \end{array}\right..$$

(8)

3.1.3. Attack Layer

The attack layer lies between the embedding and the extraction networks and operates when ReDMark is trained. The attack layer itself is not trained. By simulating possible attacks on the stego-image and feeding the attacked image to the extraction network, the network can be trained to extract the watermark from the degraded image. Various attacks can be simulated in the attack layer. In the attack layer of ReDMark [11], three networks were implemented according to the type of attacks: a GT-Net (Gaussian-trained network), a JT-Net (JPEG-trained network), and a MT-Net (multi-attack-trained network).

In ReDMark, the quantization process is approximated using the quantization table $T_Q(u,v)$ and uniform noise $ϵ$. Let $x(u,v)$ represent the DCT coefficients of an $8\times 8$-pixel block of a stego-image, and the quantized DCT coefficients $z(u,v)$ are given as

$$\begin{array}{ccc}\hfill z(u,v)& =& \left(\frac{x(u,v)}{T_Q(u,v)}+ϵ\right)T_Q(u,v)\hfill \end{array}$$

(9)

$$\begin{array}{ccc}& =& x(u,v)+T_Q(u,v)ϵ,\hfill \end{array}$$

(10)

where $ϵ$ represents noise subject to a uniform distribution in the interval $\left[-0.5,0.5\right]$. In other words, the quantization process is equivalent to adding a uniform noise $ϵ$ proportional to the quantization table $T_Q(u,v)$.

4. Proposed Method

5. Computer Simulations

5.2. Evaluation of the Proposed Attack Layer

We compared the JT-Net in the ReDMark, our previous model and the proposed quantization table-based QAF network (QT-QAF-Net) based on the image quality of stego-images and the BER of watermarks extracted from stego-images after the JPEG compression. Note that our previous model [14] was the AE-based model with the quantization table quantized by constant intensity. For comparison, we use an improved model of the CNN-based QAF network with a constant intensity quantisation table (constant QAF-Net). The only difference between the QT-QAF-Net and the constant QAF-Net are the values of the quantisation table.

5.2.1. Experimental Conditions

The training and test images were selected as they were in ReDMark [11]. The training images were 50,000 images of $32\times 32$ pixels from CIFAR10 [21] ($H=32,W=32$). An $h\times w$-bit watermark was embedded, where $h=4,w=4$. The watermark was randomly generated. Therefore, the amount of watermark embedded per pixel was approximately $0.0156$ bits per pixel. In the parameters used for training, the block height and width sizes were set to $M=8$ and $N=8$, respectively. The parameter of the loss function (20) was set to $\gamma =0.75$, the number of learning epochs was set to 100, and the mini-batch size was set to 32. For the parameters of SGD, the training rate was set to ${10}^{-4}$, and the moment was set to $0.98$. For training the proposed attack layer, the gradient of the QAF (13) was set to $\beta =1000$ and the number of the hyperbolic tangent functions was set to $n=500$. In the attack layer of JT-Net and the proposed method, the quantization level was set to $Q=70$, and the quantization table $T_{70}$ was used. The embedding, attack, and extraction networks were all connected, and the network was trained using the training images and watermarks. Here, the embedding intensity was fixed at $\alpha =1$.

Figure 11. Image quality of stego-images (tained by $Q=70$)

Figure 11. Image quality of stego-images (tained by $Q=70$)

For testing, 49 images of $512\times 512$ pixels from the University of Granada were used. These images were divided into $32\times 32$ pixel subimages, and the embedding process was performed on each of them. The 256 subimages were given to the network as test images. Meanwhile, a $32\times 32$-bit watermark was randomly generated. One watermark was embedded four times in one image. That is, the watermark was divided into $4\times 4$-bit subwatermarks, and finally each subwatermark was embedded in one subimage. An estimated watermark was determined by bit-by-bit majority voting because the same watermark was embedded four times in one image.

As stated in Section 3.1, the attack layer was not used during testing. The test images and the watermarks were used to output stego-images in the embedding network. Here, the embedding intensity was set to values from $\alpha =0.5$ to $1.0$. The stego-image is published. Subsequently, we assume that it was JPEG compressed by some JPEG tool with quantization levels $Q=10,20,\cdots ,90$. The compressed stego-images were input to the extraction network, and the estimated watermarks were output. These networks were trained 10 times with different initial weights for the comparison of our network with the JT-Net on image quality and BER. The mean and standard deviation of structural similarity index measures (SSIMs), PSNRs, and BERs were calculated.

5.2.2. Evaluation of the Image Quality

The image quality of the stego-images obtained from the JT-Net, the constant QAF-Net and QT-QAF-Net was evaluated using the SSIMs and PSNRs. The image quality of the stego-image ${\mathit{I}}_{st}$ against the original image $\mathit{I}$ can be expressed as $\mathrm{SSIM}\left({\mathit{I}}_{st},\mathit{I}\right)$ by (17) and $\mathrm{PSNR}\left({\mathit{I}}_{st}|\mathit{I}\right)$ by (23). Figure 11 shows the SSIM and PSNR. The horizontal and vertical axes represent the embedding intensity $\alpha$ and SSIM or PSNR, respectively. The error bars represent the standard deviation of the SSIMs and PSNRs. Embedding the watermark strongly causes degradation in the image quality. Therefore, the SSIM and PSNR decreased as the intensity $\alpha$ increased. The image quality of the proposed QT-QAF-Net was higher than that of the other two networks. In other words, our network can reduce the degradation of image quality even with the same embedding intensity.

As the images were processed block by block, they were visually checked for block artefacts. Three images selected from the dataset were cropped to $128\times 128$-pixel size as shown in the Figure 12. These images were generated from the proposed network trained with embedding intensity $\alpha =1.0$. The images were not reduced in size when displayed. Few noticeable artifacts were observed.

Figure 12. Images cropped to $128\times 128$ pixels.

Figure 12. Images cropped to $128\times 128$ pixels.

Figure 13. BER for embedding intensity $\alpha$

Figure 13. BER for embedding intensity $\alpha$

5.2.3. Evaluation of the BER

The robustness of our network against the JPEG compression was evaluated. The estimated watermark obtained by (8) was evaluated by using the BER. The BER of the estimated watermark ${\mathit{p}}_e$ can be defined by

$$\mathrm{BER}=\frac{1}{hw}\sum _{i=0}^{h-1}\sum _{j=0}^{w-1}p(i,j)\oplus p_e(i,j),$$

(25)

where $\mathit{p}$ is the original watermark, and ⊕ represents the exclusive OR.

First, we compared the robustness of our network with that of the JT-Net and constant QAF-Net using the same embedding intensity $\alpha$. Figure 13 shows the BER of the estimated watermark for the embedding intensity $\alpha$. The horizontal and vertical axes represent the intensity $\alpha$ and BER, respectively. For different compression levels Q, the dashed lines with circles represent the BER for the QT-QAF-Net, the solid lines with squares represent the BER for the JT-Net and the dotted lines with triangles represent the BER for the constant QAF-Net. When the watermark was strongly embedded, it was extracted correctly. Therefore, the BER decreased as the intensity $\alpha$ increased. The lowest BER was obtained when $\alpha =1.0$. At compression level $Q\le 70$, the BER of the proposed network was lower than that of the JT-Net. Also, at $Q=80$, they had almost the same BER. Furthermore, at $Q=90$, the BER of our network was larger than that of the JT-Net. The BER of the constant QAF-Net was always larger than that of the other two networks.

Even with the same embedding intensity, the image quality of our network differs from that of the JT-Net and the constant QAF-Net. Therefore, we next adjusted the embedding intensity so that the PSNRs of these three networks were approximately the same, and we compared the BERs of the networks under this condition. Figure 14 shows the histograms of PSNRs for the QT-QAF-Net with embedding intensity $\alpha =1.0$. (a) shows the histograms for the embedding intensity $\alpha =1.0$ for the JT-Net and $\alpha =0.55$ for the constant QAF-Net, respectively. (b) shows histograms for these embedding intensities $\alpha =0.95$ and $\alpha =0.50$, respectively. The intensities of the three networks were chosen so that the histograms shown in (b) look similar. To measure the robustness against JPEG compression, we set the intensity $\alpha$ to ensure that the PSNR obtained from these networks is approximately the same. Specifically, we set the intensity for the JT-Net and QT-QAF-Net to $\alpha =0.95$ (average PSNR$=37.58$ dB) and $\alpha =1.0$ (average PSNR$=37.81$ dB), respectively. Figure 15 is the BER for the compression level Q. The error bars are the standard deviation of the BERs. The BER of the estimated watermark for QT-QAF-Net was lower than that for the JT-Net and constant QAF-Net. Thus, we can say that our network can generate watermarks with fewer errors under the given PSNR.

6. Conclusion

References

1. Wan, W.; Wang, J.; Zhang, Y.; Li, J.; Yu, H.; Sun, J. A comprehensive survey on robust image watermarking. Neurocomputing 2022, 488, 226–247, . [CrossRef]
2. M. Vafaei, H. M.-Nasab and H. Pourghassem, “A new robust blind watermarking method based on neural networks in wavelet transform domain,”World Applied Sciences Journal, vol. 22, no. 11, pp. 1572–1580, 2013.
3. Sy, N.C.; Kha, H.H.; Hoang, N.M. An Efficient Robust Blind Watermarking Method Based on Convolution Neural Networks in Wavelet Transform Domain. Int. J. Mach. Learn. Comput. 2020, 10, 675–684, . [CrossRef]
4. He, D.; Xu, K.; Wang, D. Design of multi-scale receptive field convolutional neural network for surface inspection of hot rolled steels. Image Vis. Comput. 2019, 89, 12–20, . [CrossRef]
5. Singh, H.K.; Singh, A.K. Digital image watermarking using deep learning. Multimedia Tools Appl. 2023, 83, 2979–2994, . [CrossRef]
6. Hamamoto, I.; Kawamura, M. Image Watermarking Technique Using Embedder and Extractor Neural Networks. IEICE Trans. Inf. Syst. 2019, E102.D, 19–30, . [CrossRef]
7. M. Jamali , N. Karimi, P. Khadivi, S. Shirani, S. Samavi “Robust watermarking using diffusion of logo into auto-encoder feature maps,” Multimedia Tools and Applications, vol. 82, pp.45175–45201, 2023.
8. J. Zhu, R. Kaplan, J. Johnson, L. Fei-Fei, “HiDDeN: hiding data with deep networks,” European Conference on Computer Vision, 2018.
9. Zhao, Y.; Wang, C.; Zhou, X.; Qin, Z. DARI-Mark: Deep Learning and Attention Network for Robust Image Watermarking. Mathematics 2022, 11, 209, . [CrossRef]
10. Hamamoto, I.; Kawamura, M. Neural Watermarking Method Including an Attack Simulator against Rotation and Compression Attacks. IEICE Trans. Inf. Syst. 2020, E103.D, 33–41, . [CrossRef]
11. Ahmadi, M.; Norouzi, A.; Karimi, N.; Samavi, S.; Emami, A. ReDMark: Framework for residual diffusion watermarking based on deep networks. Expert Syst. Appl. 2019, 146, 113157, . [CrossRef]
12. R. Shin, D. Song, “Jpeg-resistant adversarial images,” NIPS 2017 workshop on machine learning and computer security, vol. 1, pp. 8–13, 2017.
13. Mareen, H.; Antchougov, L.; Van Wallendael, G.; Lambert, P. Blind deep-learning-based image watermarking robust against geometric transformations, 2024 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, USA, 2024, pp. 1-2, 2024.
14. S. Yamauchi, M. Kawamura, “Neural network based watermarking trained with quantized activation function,” Asia Pacific Signal and Information Processing Association Annual Summit and Conference 2022, pp. 1608–1613, 2022.
15. C. Guo, M. Rana, M. Cisse, L. van der Maaten, “Countering adversarial images using input transformations,” International Conference on Learning Representations, 2018.
16. Independent JPEG group, http://www.ijg.org/, (Accessed 2023-05-21).
17. D. Clevert, T. Unterthiner, S. Hochreiter, “Fast and accurate deep network learning by exponential linear units (ELUs),” arXiv preprint arXiv:1511.07289, 2015.
18. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Trans. Image Process. 2004, 13, 600–612, . [CrossRef]
19. D. E. Rumelhart, G. E. Hinton, R. J.Williams, “Learning representations by back-propagating errors,” Nature, vol. 323, no. 6, pp. 533–536, 1986.
20. Computer vision group at the University of Granada, dataset of standard 512 × 512 grayscale test images, http://decsai.ugr.es/cvg/CG/base.htm, (Accessed 2023-06-09).
21. A. Krizhevsky, V. Nair, G. Hinton, The CIFAR-10 dataset, https://www.cs.toronto.edu/ kriz/cifar.html, (Accessed 2023-06-09).
22. Lin, R.; Liu, S.; Jiang, J.; Li, S.; Li, C.; Kuo, C.-C.J. Recovering sign bits of DCT coefficients in digital images as an optimization problem. J. Vis. Commun. Image Represent. 2024, 98, . [CrossRef]