Language model training is slow, even when your model is not very large. This is because you need to train the model with a large dataset and there is a large vocabulary. Therefore, it needs many training steps for the model to converge. However, there are some techniques known to speed up the training process. In this article, you will learn about them. In particular, you will learn about: Using optimizers Using learning rate schedulers Other techniques for better convergence or reduced memory consumption Let’s get started. How to Speed-Up Training of Language ModelsPhoto by Emma Fabbri. Some rights reserved. Overview This article is divided into four parts; they are: Optimizers for Training Language Models Learning Rate Schedulers Sequence Length Scheduling Other Techniques to Help Training Deep Learning Models Optimizers for Training Language Models Adam has been the most popular optimizer for training deep learning models. Unlike SGD and RMSProp, Adam uses both the first and second moment of the gradient to update the parameters. Using the second moment can help the model converge faster and more stably, at the expense of using more memory. However, when training language models nowadays, you will usually use AdamW, the Adam optimizer with weight decay. Weight decay is a regularization technique to prevent overfitting. It usually involves adding a small penalty to the loss function. But in AdamW, the weight decay is applied directly to the weights instead. This is believed to be more stable because the regularization term is decoupled from the calculated gradient. It is also more robust to hyperparameter tuning, as the effect of the regularization term is applied explicitly to the weight update. In formula, AdamW weight update algorithm is as follows: $$\begin{aligned}g_t &= \nabla_\theta L(\theta_{t-1}) \\m_t &= \beta_1 m_{t-1} + (1 – \beta_1) g_t \\v_t &= \beta_2 v_{t-1} + (1 – \beta_2) g_t^2 \\\hat{m_t} &= m_t / (1 – \beta_1^t) \\\hat{v_t} &= v_t / (1 – \beta_2^t) \\\theta_t &= \theta_{t-1} – \alpha \Big( \frac{\hat{m_t}}{\sqrt{\hat{v_t}} + \epsilon} + \lambda \theta_{t-1} \Big)\end{aligned}$$ The model weight at step $t$ is denoted by $\theta_t$. The $g_t$ is the computed gradient from the loss function $L$, and $g_t^2$ is the elementwise square of the gradient. The $m_t$ and $v_t$ are the moving average of the first and second moment of the gradient, respectively. Learning rate $\alpha$, weight decay $\lambda$, and moving average decay rates $\beta_1$ and $\beta_2$ are hyperparameters. A small value $\epsilon$ is used to avoid division by zero. A common choice would be $\beta_1 = 0.9$, $\beta_2 = 0.999$, $\epsilon = 10^{-8}$, and $\lambda = 0.1$. The key of AdamW is the $\lambda \theta_{t-1}$ term in the gradient update, instead of in the loss function. AdamW is not the only choice of optimizer. Some newer optimizers have been proposed recently, such as Lion, SOAP, and AdEMAMix. You can see the paper Benchmarking Optimizers for Large Language Model Pretraining for a summary. Learning Rate Schedulers A learning rate scheduler is used to adjust the learning rate during training. Usually, you would prefer a larger learning rate for the early training steps and reduce the learning rate as training progresses to help the model converge. You can add a warm-up period to increase the learning rate from a small value to the peak over a short period (usually 0.1% to 2% of total steps), then the learning rate is decreased over the remaining training steps. A warm-up period usually starts with a near-zero learning rate and increases linearly to the peak learning rate. A model starts with randomized initial weights. Starting with a large learning rate can cause poor convergence, especially for big models, large batches, and adaptive optimizers. You can see the need for warm-up from the equations above. Assume the model is uncalibrated; the loss may vary greatly between subsequent steps. Then the first and second moments $m_t$ and $v_t$ will be fluctuating greatly, and the gradient update $\theta_t – \theta_{t-1}$ will also be fluctuating greatly. Hence, you would prefer the loss to be stable and move slowly so that AdamW can build a reliable running average. This can be easily achieved if $\alpha$ is small. At the learning rate reduction phase, there are a few choices: cosine decay: $LR = LR_{\max} \cdot \frac12 \Big(1 + \cos \frac{\pi t}{T}\Big)$ square-root decay: $LR = LR_{\max} \cdot \sqrt{\frac{T – t}{T}}$ linear decay: $LR = LR_{\max} \cdot \frac{T – t}{T}$ Plot of the three decay functions A large learning rate can help the model converge faster while a small learning rate can help the model stabilize. Therefore, you want the learning rate to be large at the beginning when the model is still uncalibrated, but small at the end when the model is close to its optimal state. All decay schemes above can achieve this, but you would not want the learning rate to become “too small too soon” or “too large too late”. Cosine decay is the most popular choice because it drops the learning rate more slowly at the beginning and stays longer at a low learning rate near the end, which are desirable properties to help the model converge faster and stabilize respectively. n PyTorch, you have the CosineAnnealingLR scheduler to implement cosine decay. For the warm-up period, you need to combine with the LinearLR scheduler. Below is an example of the training loop using AdamW, CosineAnnealingLR, and LinearLR: import torch import torch.nn as nn import torch.optim as optim from torch.optim.lr_scheduler import LinearLR, CosineAnnealingLR, SequentialLR # Example setup model = torch.nn.Linear(10, 1) X, y = torch.randn(5, 10), torch.randn(5) loss_fn = nn.MSELoss() optimizer = optim.AdamW(model.parameters(), lr=1e-2, betas=(0.9, 0.999), eps=1e-8, weight_decay=0.1) # Define learning rate schedulers warmup_steps = 10 total_steps = 100 min_lr = 1e-4 warmup_lr = LinearLR(optimizer, start_factor=0.1, end_factor=1.0, total_iters=warmup_steps) cosine_lr = CosineAnnealingLR(optimizer, T_max=total_steps – warmup_steps, eta_min=min_lr) combined_lr = SequentialLR(optimizer, schedulers=[warmup_lr, cosine_lr], milestones=[warmup_steps]) # Training loop for step in range(total_steps): # train one epoch y_pred = model(X) loss = loss_fn(y_pred, y) # print loss and learning rate print(f”Step {step+1}/{total_steps}: loss {loss.item():.4f}, lr {combined_lr.get_last_lr()[0]:.4f}”) # backpropagate and update weights optimizer.zero_grad()

New Delhi: India Inc. is rapidly scaling AI, fueled by global tailwinds, competition and advances in GenAI technologies, a report said on Wednesday, noting that AI now cuts across customer engagement, operational optimisation and mission-critical processes in multiple sectors. Yet adoption remains fragmented, with only 15 per cent of organisations having extensive enterprise-wide AI deployment. “While AI will continue growing, oversight is not keeping pace with it. In many organisations, AI infrastructure is expanding faster than the governance, security and ethical safeguards needed, creating widening gaps in accountability and risk management,” Alvarez & Marsal (A&M), a global professional services firm, said in its report. Meanwhile, governance maturity remains limited despite rising usage. While 60 per cent of organisations have introduced basic governance or acceptable-use policies, only 19 per cent have carried out detailed risk assessments, and 81 per cent still lack full visibility of how their AI systems are monitored or governed, the report noted. Add Zee News as a Preferred Source With many AI initiatives developed in silos, accountability and standards vary widely, especially when third-party and in-house models coexist. The report highlighted the need for integrated, organisation-wide governance frameworks that embed transparency, oversight and clear role ownership. “AI is now embedded deeper into business processes and decision systems than ever before. India’s AI opportunity is substantial, but its long-term gains depend on how effectively organisations govern and secure the systems they deploy,” said Dhruv Phophalia, MD and India Lead – Disputes and Investigations, Alvarez & Marsal. Those who invest early in these foundations will be best placed to unlock the full economic and competitive potential of AI, he added. According to the report, responsible AI principles are widely acknowledged; however, their implementation remains limited. Fewer than 20 per cent of organisations have deployed mechanisms for explainability, bias detection or fairness, and 60 per cent lack any formal process to validate model integrity. Data governance shows similar gaps, with only 26 per cent having integrated data masking and PII-scanning within AI workflows, and 60 per cent perform no structured dataset validation. The report also highlighted that as more complex AI models go into production, security across the AI lifecycle will be imperative. While 52 per cent of enterprises have secure development environments with basic controls, fewer than 30 per cent conduct penetration testing or red-teaming, and only 19 per cent have safeguards to detect data poisoning during model training.

New Delhi: The total number of Internet subscribers in India increased from 1002.85 million at the end of the April-June quarter (Q1 FY26) to 1017.81 million at the end of the July-September period (Q2 FY26), registering a quarterly growth of 1.49 per cent, data from Telecom Regulatory Authority of India (TRAI) showed on Wednesday. As per the data, out of 1,070.81 million internet subscribers, the number of Wired Internet subscribers is 44.42 million, and the number of Wireless Internet subscribers is 973.39 million. Meanwhile, the broadband Internet subscriber base increased by 1.63 per cent from 979.71 million at the end of June to 995.63 million at the end of September this year. The narrowband Internet subscriber base decreased from 23.14 million in the June quarter to 22.18 million at the end of the September quarter. Add Zee News as a Preferred Source At the same time, wireline subscribers decreased from 47.49 million at the end of the April-June quarter to 46.61 million at the end of the September quarter, with a quarterly rate of decline of 1.84 per cent. On a year-on-year (YoY) basis, wireline subscriptions increased by 26.21 per cent at the end of the July-September quarter. Wireline Tele-density decreased from 3.36 per cent at the end of Q1 FY26 to 3.29 per cent at the end of Q2 FY26, with a quarterly rate of decline of 2.06 per cent. Meanwhile, the monthly average revenue per user (ARPU) for wireless service increased by 2.34 per cent, from Rs 186.62 in the first quarter to Rs 190.99 in the second quarter of this fiscal. Monthly ARPU for wireless service increased by 10.67 per cent YoY in this quarter as well. The ARPU per month for the pre-paid segment is Rs 189.69, and the ARPU per month for the post-paid segment is Rs 204.55 for the quarter under review. On an all-India average, the overall MOU per month decreased by 0.10 per cent from 1006 in the April-June period to 1005 at the end of the July-September period. The total Internet subscriber base comprises a Broadband Internet subscriber base of 995.63 million and a Narrowband Internet subscriber base of 22.18 million.

Redmi 15C 5G Price In India: Xiaomi has launched the Redmi 15C 5G smartphone in the Indian market, a day after Vivo introduced the Vivo X300 series in the country. The smartphone runs Android 15 with Xiaomi HyperOS 2 and will receive 2 years of software updates and 4 years of security updates. The Redmi 15C 5G arrives as a budget handset and the successor to the Redmi 14C, which was introduced in January this year. It comes in Midnight Black, Moonlight Blue, and Dusk Purple colour options. The smartphone is offered in 4GB RAM + 128GB, 6GB RAM + 128GB and 8GB RAM + 128GB storage variants. Redmi 15C 5G Specifications Add Zee News as a Preferred Source The smartphone features a 6.9-inch HD+ (720 × 1,600 pixels) AdaptiveSync display with a 120Hz refresh rate and 240Hz touch sampling rate for smooth visuals and responsive touch. It is powered by a 6,000mAh battery that supports 33W fast charging. The phone measures 171.56 × 79.47 × 8.05mm and weighs 211g. On the photography front, the smartphone supports a 50MP AI dual rear camera with an f/1.8 aperture, along with an 8MP front camera for selfies and video calls. On the connectivity front, the smartphone supports 5G, 4G, Wi-Fi, Bluetooth 5.4, GPS, and a USB Type-C port. (Also Read: OPPO A6x 5G Launched In India With MediaTek Dimensity 6300 Chipset; Check Camera, Battery, Display, Price, Availability And Other Features) Redmi 15C 5G Price In India And Sale Date The Redmi 15C 5G starts at Rs 12,499 for the base model with 4GB RAM and 128GB storage. It also comes in 6GB and 8GB RAM variants, priced at Rs 13,999 and Rs 15,499. The phone will be available for purchase on Amazon and the Xiaomi India website starting December 11. 

Sanchar Saathi App: Since its launch on January 17 this year, the Sanchar Saathi mobile app has seen more than 1.4 crore downloads, and have successfully blocked over 42 lakh stolen or lost mobile devices, official data showed on Tuesday. While 26 lakh lost/stolen mobile phones were traced, 7.23 lakh have successfully been returned with the help of Sanchar Saathi app which is a fully voluntary, user-driven platform and privacy-first app and activates only with user consent. Sanchar Saathi app puts citizens first and protects their privacy at every step. It works only with user’s consent and gives full control over its activation and use, according to the data. Add Zee News as a Preferred Source It activates only after user chooses to register and the user may activate, deactivate, or delete it any time. The app has been designed to strengthen India’s cybersecurity without compromising privacy. Rising cyber threats have made safeguarding mobile users a pressing national concern. According to the Indian Computer Emergency Response Team (CERT-In), cybercrime incidents surged from 15,92,917 in 2023 to 20,41,360 in 2024. Digital Arrest Scams and related cybercrimes reported on the National Cyber Crime Reporting Portal alone totalled 1,23,672 in 2024, with 17,718 cases already reported by February 2025. In response to these escalating threats, the Department of Telecommunications (DoT) introduced the Sanchar Saathi mobile app — a citizen-centric tool that brings robust security features and fraud-reporting capabilities directly to users’ smartphones. The app complements the existing Sanchar Saathi portal by providing convenient, on-the-go protection against identity theft, forged KYC, device theft, banking fraud, and other cyber risks. To strengthen the initiative, the Department of Telecommunications has issued directions, mandating mobile manufacturers and importers to facilitate the availability and accessibility of the Sanchar Saathi app on devices for users in India. By empowering citizens with easy-to-use tools and real-time access to vital security features, the Sanchar Saathi mobile app represents a timely and effective response to India’s growing cybercrime challenges. The application is available in Hindi and 21 other regional languages, making it inclusive and accessible across the country. Sanchar Saathi prioritises user privacy and collects only the minimum personal information necessary to provide services.

OPPO A6x 5G Price In India: Chinese smartphone maker OPPO has launched its budget-friendly OPPO A6x 5G smartphone in India. The newly launched device is the successor to the A5x 5G, which was introduced earlier this year. The OPPO A6x 5G runs on ColorOS 15, offering several new features designed to enhance the overall user experience, including the Luminous Rendering Engine. The OPPO A6x 5G comes in two elegant colour options, Ice Blue and Olive Green, giving users a stylish and refreshing look. It is available in three storage variants to suit different usage needs: 4GB RAM with 64GB storage, 4GB RAM with 128GB storage, and 6GB RAM with 128GB storage, providing a smoother performance and more space for apps and media. OPPO A6x 5G Specifications Add Zee News as a Preferred Source The smartphone sports a 6.75-inch LCD display with an HD+ resolution of 720×1,570 pixels, offering up to a 120Hz refresh rate, 256ppi pixel density, up to a 240Hz touch sampling rate, and peak brightness of up to 1,125 nits. The device is powered by an octa-core MediaTek Dimensity 6300 chipset paired with an ARM Mali-G57 MC2 GPU for smooth performance. The smartphone houses a large 6,500mAh battery that supports 45W wired SuperVOOC fast charging. On the photography front, the OPPO A6x 5G features a 13-megapixel primary rear camera with a 77-degree field of view and autofocus. For selfies and video chats, there is 5-megapixel shooter at the front, offering the same 77-degree field of view for selfies and video calls. On the connectivity front, the smartphone options include 5G, 4G LTE, Wi-Fi 5, Bluetooth 5.4, a USB Type-C port, and a 3.5mm audio jack. The handset measures 166.6×78.5×8.6mm and weighs around 212 grams. (Also Read: Vivo X300 Pro, Vivo X300 Launched In India With MediaTek Dimensity 9500 Chipset; Check Camera, Display, Battery Price, Sale And Other Specs) OPPO A6x 5G Price In India And Availability The OPPO A6x 5G comes in three variants which include the 4GB RAM with 64GB storage priced at Rs 12,499, 4GB RAM with 128GB storage priced at Rs 13,499, and 6GB RAM with 128GB storage priced at Rs 14,999. Consumers can purchase the smartphone across major online platforms such as Amazon, Flipkart, and the OPPO Store, as well as mainline retail outlets. Customers can also avail a three-month no-cost EMI option on select bank cards. 

OnePlus 15R Price In India: Chinese smartphone brand OnePlus is set to launch its next budget flagship, the OnePlus 15R, in the Indian market. The company has announced that the OnePlus 15R will make its official debut on December 17. Meanwhile, the brand has published the first visuals of the device on its website, showcasing a clean, flat-edged design that closely aligns with the premium OnePlus 15 series. The OnePlus 15R will come in a classic Black and a fresh Green finish colour options. Adding further, the OnePlus 15R smartphone is expected to include the customizable Plus key seen on recent OnePlus phones. Certification details from China suggest it may come with an IP68 rating, and possibly even IP69 water and dust protection. OnePlus 15R Specifications (Expected) Add Zee News as a Preferred Source If the OnePlus 15R mirrors the OnePlus Ace 6 specs, the smartphone is expected to feature a large 6.83-inch flat AMOLED display with a 1.5K resolution, a smooth 165Hz refresh rate, and an impressive 5,000-nit peak brightness. The device may powered by a Snapdragon 8 Elite chipset, paired with up to 16GB LPDDR5X RAM and up to 512GB UFS 4.1 storage.  For photography, the phone may offer a 50MP OIS-enabled main camera, an 8MP ultra-wide lens, and a 16MP front camera. The phone is likely to run ColorOS 16 based on Android 16. It is packed by a massive 7,800mAh battery with 120W SuperVOOC fast charging. It is also said to come with IP66/68/69/69K dust and water protection. (Also Read: iQOO 15 With 7,000mAh Battery And Triple-Camera Setup Goes on Sale in India: Check Display, Price, Availability, And Bank Discount) OnePlus 15R Price (Expected) If the OnePlus 15R follows the pricing of the OnePlus Ace 6 in China, it may adopt a similar structure. The Ace 6 starts at CNY 2599 (approximately Rs 32,300) for the 12GB 256GB variant. The other configurations, 16GB 256GB, 12GB 512GB, and 16GB 512GB, are priced at CNY 2899 (about Rs 36,000), CNY 3099 (Rs 38,800), and CNY 3399 (about 42,200 rupees) respectively.  

New Delhi: The Cellular Operators Association of India (COAI) on Monday welcomed the Department of Telecommunications’ (DoT) directive mandating Subscriber Identity Module (SIM)-binding for devices for app-based communication services, saying the move will bolster national security and curb cyber fraud. “Continuous linkage ensures complete accountability and traceability for any activity undertaken by the SIM card and its associated communication app, thereby closing long-persistent gaps that have enabled anonymity and misuse,” said Lt. Gen. Dr. S.P. Kochhar, Director General, COAI. This is a much-needed initiative in ensuring consumer trust, accountability, traceability and further alignment with evolving regulatory frameworks, the release said. The association also called on the DoT to engage the Reserve Bank of India (RBI) to mandate SMS one‑time passwords (OTP) as the primary authentication factor for all financial transactions. Add Zee News as a Preferred Source “SMS OTP continues to remain the most secure, operator verified channel with guaranteed traceability. Strengthening this requirement will create a consistent and secure authentication framework across the financial ecosystem, further reducing the risk of fraud and reinforcing consumer trust,” the statement said. App based communication services must remain continuously linked to the SIM card, which is associated with the mobile number used for identification of customers/users or for provisioning or delivery of services. The directive mandated that the user’s subscriber identity module (SIM) used at registration must be bound to the services of web-based platforms such as WhatsApp, Telegram, Signal, Arattai, Snapchat, Sharechat, and others. As the service must remain tied to the SIM in the phone, WhatsApp Web and similar web platforms are forced to log users out every six hours once the rule is implemented. Each web-based platform must submit a compliance report within four months. The change will disrupt the seamless multi‑device experience many gained by keeping WhatsApp Web running throughout the workday.

Elon Musk On ‘WTF’ Podcast: A new teaser of Elon Musk’s appearance on Zerodha co-founder Nikhil Kamath’s “WTF is” podcast has sparked fresh excitement online, with Kamath releasing a second black-and-white clip that blends humour with deeper reflections from the Tesla and SpaceX chief. The video, posted on X, opens with a light moment as Kamath comments on Musk’s physique, saying he looks “bigger and bulkier” than expected. Musk laughs it off before the conversation shifts to his long-running fascination with the letter ‘X’. When Kamath asks why he likes the letter so much, Musk responds jokingly, “Sometimes I wonder what’s wrong with me.” The teaser then takes a pop-culture turn when Kamath asks which character from “The Matrix” Musk would want to be. Musk replies, “Hopefully not Agent Smith? He’s my hero,” keeping the tone playful. Kamath shared the clip with the caption, “Here’s to delaying gratification. Add Zee News as a Preferred Source “Here’s to delaying gratification. The future belongs to the patient,” hinting at the theme of the full conversation while offering few concrete details beyond its “coming soon” promise. The future belongs to the patient,” hinting at the broader themes of perseverance and long-term thinking that may feature in the full conversation. This comes just a day after Kamath released the first teaser, a 39-second silent clip showing him and Musk sitting together in what appears to be an industrial setting. The two sip coffee and exchange smiles without any dialogue, while a SpaceX logo on Kamath’s mug prompted speculation about the topics and setting of the full episode. List Of Guest On Nikhil Kamath’s Podcast Kamath’s podcast has gained global recognition, attracting prominent figures from technology, business, and politics. Notable guests have included Microsoft co-founder Bill Gates, investor Vinod Khosla, and Perplexity AI CEO Aravind Srinivas, highlighting its growing influence and reach worldwide. 

Mobile Restart Android And Laptop: In today’s digital world, our smartphones, laptops, and PCs run many apps, background tasks, and constant software updates. With so much happening, we often forget one of the easiest ways to keep our devices healthy: restarting them. Whether you use your phone for work or your laptop or PC for study, gaming, or entertainment, a quick restart can make a big difference. Let’s take a closer look at why restarting your device is more useful than you may realize. Clears Memory And Speeds Up When you restart your device, it clears memory, temporary files, and all apps running in the background that quietly use resources. This refreshes the system, reduces lag, makes apps open faster, improves multitasking, and helps your phone or laptop feel almost like new. Add Zee News as a Preferred Source Improves Battery Life And Reduces Heating Background apps that keep running can drain your battery and make your device hot. Restarting your phone or laptop stops these hidden tasks, reduces the load on the processor, improves battery life, and helps it stay cooler during heavy use. Fixes Common Glitches And Connectivity Issues Many common issues like Wi Fi not connecting, Bluetooth problems, slow internet, or apps crashing can be fixed with a simple restart. Rebooting refreshes system and network services, giving your device a clean start and clearing minor glitches. Enhances Security And Applies Updates Experts and IT teams recommend restarting devices regularly because it can stop certain memory-based cyberattacks and ensures security updates install correctly. On laptops, many operating system and antivirus updates only complete after a restart, keeping your data safer from the latest threats. (Also Read: Samsung Galaxy Tab A11+ Tablet Launched In India With MediaTek MT8775 Processor; Check Camera, Display, Battery, Price And Other Features) How To Restart Your Mobile Phone Step 1: Press and hold the Power button (or Power + Volume Down on some phones) until the power menu appears.​ Step 2: Tap Restart or Reboot and wait for the phone to turn off and turn on again.​ Step 3: If the phone is frozen, hold Power and Volume Down together for 10–20 seconds until it. How To Restart Windows Laptop Or PC Step 1: Save your work and close open apps first.​ Step 2: Click the Start button, select the Power icon, then choose Restart (or Update and Restart if shown).​ Step 3: You can also press Ctrl + Alt + Delete, click the power icon at the bottom right, and choose Restart if the system is lagging. Restart your phone and laptop at least once a week. Regular restart help keep performance smooth, improve battery life, reduce glitches, and maintain system stability, ensuring your devices run efficiently and stay healthy over time.