Malaysian Journal of Mathematical Sciences, March 2026, Vol. 20, No. 1
Optimizing Variance for Reliability Decryption in NTRUEncrypt
Daud, M. A., Kamarulhaili, H., Mandangan, A., and Asbullah, M. A.
Corresponding Email: azlan.daud@ums.edu.my, hailiza@usm.my
Received date: 12 August 2024
Accepted date: 5 August 2025
Abstract:
This paper investigates factors influencing the polynomial coefficients of
\begin{align*}
a(x) = p \cdot r(x) * g(x) + m(x) * f(x),
\end{align*}
ensuring they remain within the interval \(\left(-\dfrac{q - 1}{2},\ \dfrac{q - 1}{2}\right]\) in NTRUEncrypt. The study highlights the significance of coefficient distribution and randomness. Through probability theory and statistical analysis, critical conditions for reliable decryption are identified. Findings demonstrate that selecting appropriate parameters ensures polynomial coefficients stay within the required bounds for successful decryption. By developing a detailed framework, the study focuses on optimizing parameter selection in NTRUEncrypt. The framework strengthens the theoretical and practical resilience of NTRUEncrypt against decryption failures, providing a deeper understanding of its decryption dynamics and enhancing its reliability in cryptographic applications.
Keywords: post quantum cryptography; ntruencrypt; decryption success; public key structures; coefficients probability distribution; coefficient variance.
