“Where does Spring Boot look for @Value("${my.property:DEFAULT_VALUE}"), and in what order?”

At first, it might sound straightforward, but it actually tests a developer’s understanding of how Spring Boot resolves configuration properties — a concept that directly impacts how applications behave across environments.

Understanding the configuration hierarchy in Spring Boot is essential because it helps you:

• Debug issues when the wrong value is picked up.

• Control how environment variables and external configurations override defaults.

• Manage different environments such as development, staging, and production in CI/CD pipelines.

In this article, we’ll look at how @Value works, the exact order in which Spring Boot looks for property values, and why this knowledge is crucial for building predictable and maintainable applications.

Understanding the @Value Annotation

The @Value annotation in Spring Boot is used to inject values into fields directly from property sources such as application.properties, application.yml, environment variables, or command-line arguments. It allows you to bind configuration values to variables in your code without hardcoding them.

Here’s a simple example:

@Value("${my.property:DEFAULT_VALUE}")
private String myProperty;

In this example:

• my.property is the key whose value Spring Boot will try to resolve.

• DEFAULT_VALUE is the fallback value that will be used if the property is not found in any configuration source.

This default value mechanism is very useful. It ensures your application does not fail to start even if a configuration is missing. Instead, Spring Boot will inject the specified default, keeping your application stable and predictable.

Using @Value is a simple way to access configuration properties for small or single-value injections. For more complex or grouped configurations, developers often use @ConfigurationProperties, which we’ll discuss later.

Where Does Spring Boot Look for Properties (Order of Resolution)

When you use @Value or any other property injection mechanism, Spring Boot doesn’t just look in one place for configuration values. It searches across multiple sources in a specific priority order.

This order determines which value is finally injected when the same property key appears in different places. The higher the source is in the list, the greater its priority.

Here’s the general order in which Spring Boot resolves properties (based on the SpringApplication configuration sources):

Spring Boot checks these sources in order — from top to bottom — and uses the first value it finds for a given property key.

This hierarchy allows you to easily override configurations at runtime without changing your code. For example, values from command-line arguments or environment variables can override those in your application files, which is especially useful for deploying the same codebase to multiple environments like development, testing, and production.

Example Demonstration

Let’s look at a simple example to understand how Spring Boot decides which property value to use when multiple sources define the same key.

Imagine the following project structure:

src/
 └─ main/resources/
     ├─ application.properties
     ├─ application-dev.properties

Step 1: Define properties in both files

application.properties

my.property=HelloFromApplication

application-dev.properties

my.property=HelloFromDevProfile

Step 2: Use @Value in your code

@Value("${my.property:DefaultHello}")
private String message;

Step 3: Run the application with different configurations

Case 1: Default profile (no active profile)

If you run your Spring Boot app normally:

java -jar myapp.jar

Spring Boot will load application.properties and inject the value:

HelloFromApplication

Case 2: Active profile set to “dev”

If you run:

java -jar myapp.jar --spring.profiles.active=dev

Spring Boot will load both application.properties and application-dev.properties, but the profile-specific file has higher priority.
So the value becomes:

HelloFromDevProfile

Case 3: Override from the command line

Now, if you run:

java -jar myapp.jar --my.property=HelloFromCLI

Command-line arguments take the highest priority, so this overrides everything else:

HelloFromCLI

Case 4: No property found anywhere

If you remove the property from all configuration files and don’t pass it as an argument, Spring Boot uses the default value given in the code:

DefaultHello

This example shows how Spring Boot’s property resolution order allows flexible configuration management. You can define defaults in your code, keep environment-specific values in property files, and still override them at runtime when needed.

Common Pitfalls

Even though Spring Boot’s configuration system is powerful, it’s easy to run into small mistakes that cause unexpected behavior. Here are some common pitfalls developers face when working with @Value and property resolution.

1. Forgetting to Set the Active Profile

Spring Boot supports profile-specific configuration files, such as application-dev.properties or application-prod.properties.
However, these files are only used when the corresponding profile is active. If you forget to set the profile, Spring Boot will ignore those files and fall back to the default application.properties.

For example:

java -jar myapp.jar --spring.profiles.active=dev

Without this argument, your application-dev.properties file won’t be loaded at all.

2. Unintentional Property Overrides

Because Spring Boot merges multiple property sources, it’s possible to accidentally override values from one source with another.
A common case is when an environment variable or command-line argument unintentionally replaces a property defined in your configuration file.

For instance, if your system has an environment variable named MY_PROPERTY, it might override the my.property key from your .properties file. Always check the effective configuration when debugging unexpected values.

3. Confusing @Value and @ConfigurationProperties

Both @Value and @ConfigurationProperties are used to inject configuration values, but they serve different purposes.

• Use @Value when you only need a few properties.

• Use @ConfigurationProperties when dealing with a group of related configurations (for example, database settings or API credentials).

Choosing the right one helps keep your configuration clean, organized, and easy to maintain.

Interview Tip / Summary

When asked about property resolution in Spring Boot interviews, it’s not just about recalling the list — it’s about showing that you understand how and why it works that way.

Here are the key points to keep in mind:

1. Spring Boot merges multiple property sources — it doesn’t rely on a single file.

2. Higher-priority sources override lower-priority ones — values from the command line or environment variables can replace those in your application files.

3. @Value("${my.property:DEFAULT}") provides a safety net — the default value ensures your application runs even when the property is missing.

Quick Mnemonic to Remember the Order

You can remember the lookup order with this simple phrase:

“CLI → System → Env → App → Default”

Or think of it as:

Command-line arguments
↓
System properties (-D)
↓
Environment variables
↓
Application files (.properties / .yml)
↓
Default value in code

If you end your interview answer with a short, clear explanation like this:

“Spring Boot checks multiple property sources in a specific order — starting from command-line arguments, system properties, environment variables, and application files, before finally using the default value provided in code.”

—you’ll leave a confident and lasting impression.

Frequently Asked Questions (FAQ)

1. What happens if a property is missing and no default value is given?

If a property key cannot be found in any configuration source and you haven’t provided a default value in your @Value expression, Spring Boot will throw an exception during startup.
For example:

@Value("${my.property}")
private String value;

If my.property is not defined anywhere, you’ll see an error like:

Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'my.property' in value "${my.property}"

To prevent this, always provide a fallback value:

@Value("${my.property:DefaultValue}")
private String value;

This ensures your application starts smoothly, even when the property is missing.

2. How does @ConfigurationProperties differ from @Value?

While both are used to inject configuration values, they are suited to different use cases:

• @Value is ideal for single values or small configurations.
  Example:

    @Value("${server.port:8080}")
    private int port;
  

• @ConfigurationProperties is better for structured or grouped configurations, such as database or API settings. It binds multiple related properties into a single Java object and supports validation.

  Example:

    @ConfigurationProperties(prefix = "app")
    public class AppConfig {
        private String name;
        private String version;
    }
  

In short, use @Value for simple injections and @ConfigurationProperties for larger, type-safe configurations.

3. Can we override properties at runtime?

Yes, you can override properties at runtime without changing your code.
Spring Boot allows this through various property sources, such as:

• Command-line arguments

    java -jar myapp.jar --server.port=9090
  

• Environment variables

    export SERVER_PORT=9090
  

• System properties

    java -Dserver.port=9090 -jar myapp.jar
  

These dynamic overrides are particularly useful in CI/CD pipelines or cloud environments, where configurations may differ between development, staging, and production.

This comprehensive guide covers all the essential Core Java interview questions in 2025, with clear explanations, comparisons, and code examples. From OOP principles to Java 8 features, JVM internals, and practical real-world scenarios, you’ll get a complete overview of the concepts that interviewers frequently test.

Use this guide to boost your Java knowledge, strengthen problem-solving skills, and confidently tackle interviews, making sure you’re well-prepared for both theoretical and hands-on coding rounds.

Object Oriented Programming Concepts

1. What are the principles of OOP?

OOP (Object-Oriented Programming) is based on four main principles:

1. Encapsulation – Bundling data and methods that operate on that data into a single unit (class).
   Example: Private fields with public getters/setters.

2. Abstraction – Hiding internal implementation details and exposing only essential features.
   Example: Abstract classes and interfaces.

3. Inheritance – Reusing properties and methods from an existing class into a new class.
   Example: class Dog extends Animal

4. Polymorphism – The ability of objects to take multiple forms.
   Example: Method overriding where the same method behaves differently in different subclasses.

These principles together make code modular, extensible, and maintainable.

2. Difference between abstraction and encapsulation

In short:
Abstraction hides what is done, Encapsulation hides how it’s done.

3. What is inheritance? How is it implemented in Java?

Inheritance allows one class to inherit properties and behaviors from another, promoting code reuse.

In Java:

class Parent {
    void greet() { System.out.println("Hello!"); }
}

class Child extends Parent {
    void greetChild() { System.out.println("Hi from Child!"); }
}

Here, Child inherits the greet() method from Parent.

Types of inheritance in Java:

• Single

• Multilevel

• Hierarchical

Note: Java does not support multiple inheritance with classes to avoid ambiguity, but supports it with interfaces.

4. What is polymorphism? Types?

Polymorphism allows an object to behave differently based on context.

Types:

1. Compile-time polymorphism (Method Overloading):

   • Same method name, different parameter lists.

   • Resolved at compile-time.

    void show(int a) {}
    void show(String b) {}

2. Runtime polymorphism (Method Overriding):

   • Subclass provides a new implementation of a parent method.

   • Resolved at runtime using dynamic method dispatch.

Polymorphism improves flexibility and scalability in OOP systems.

5. What is method overloading vs overriding?

Example:

// Overloading
void add(int a, int b) {}
void add(double a, double b) {}

// Overriding
class A { 
    void show() {} 
}
class B extends A { 
    void show() {} 
}

6. Can we override static methods?

No.
Static methods are bound at compile-time (using class reference), not runtime (using object).

If a subclass defines a static method with the same name as the parent class’s static method, it hides the parent method — it’s not overriding.

class Parent { 
    static void greet() {} 
}
class Child extends Parent { 
    static void greet() {} 
} // Method hiding

7. Can a constructor be overridden?

No.
Constructors are not inherited, so overriding doesn’t apply.

However, constructors can be overloaded — a class can have multiple constructors with different parameter lists.

class Person {
    Person() {}
    Person(String name) {}
}

8. Why use interfaces instead of abstract classes?

Interfaces are preferred when you want to define a contract that multiple unrelated classes can implement.

Key reasons:

• Java supports multiple interface inheritance, but not multiple class inheritance.

• Promotes loose coupling — implementation can vary independently.

• Ideal for defining APIs, services, or capabilities (e.g., Comparable, Runnable).

Example:
A class can implements Serializable, Comparable but can only extends one abstract class.

9. Interface vs Abstract Class

In modern Java, interfaces are often used to define capabilities, while abstract classes define common base logic.

10. Can we create an object of abstract class or interface?

No, both cannot be instantiated directly.
They serve as blueprints for subclasses or implementing classes.

However, you can create:

• Anonymous inner classes or lambda expressions that provide concrete implementations.

Runnable r = new Runnable() {
    public void run() { System.out.println("Running..."); }
};

Here, an anonymous implementation of the interface is created.

Java Basics

1. What is the Java Virtual Machine (JVM)?

The JVM (Java Virtual Machine) is an abstract machine that executes Java bytecode.
It provides a runtime environment for Java applications and is platform-dependent.

Key responsibilities:

• Loading: Uses the ClassLoader to load class files.

• Verifying: Ensures bytecode security.

• Executing: Uses the Just-In-Time (JIT) compiler to convert bytecode to native code.

• Memory Management: Allocates and manages heap, stack, and garbage collection.

In short:
👉 Java code → compiled to bytecode → executed by JVM → runs on any OS (platform independence).

2. Explain JRE vs JDK vs JVM.

Relationship:
👉 JDK = JRE + development tools
👉 JRE = JVM + libraries

3. What is the role of main() method in Java?

main() is the entry point of any standalone Java application.
Signature:

public static void main(String[] args)

Breakdown:

• public: Accessible by JVM.

• static: JVM can call it without creating an object.

• void: Doesn’t return any value.

• String[] args: Accepts command-line arguments.

Without main(), the program doesn’t have a starting point.

4. What happens if the main() method is not static?

If main() is not static, the JVM cannot invoke it directly because no object of the class exists yet.
This leads to a runtime error like:

Error: Main method is not static in class MainClass

JVM needs main() to be static so that it can call it without object instantiation.

5. Can Java run without main()?

In modern Java (Java 7+), no — every standalone program needs a main() method.

Earlier (in applets or servlet containers), Java classes could be run without main() because:

• Applets start with the init() method.

• Servlets are loaded and managed by a container (init(), service(), destroy()).

For normal console applications → main() is mandatory.

Memory Management

1. Explain Java memory model.

The Java Memory Model (JMM) defines how Java threads interact through memory — how variables are read/written, and how visibility is ensured in concurrent execution.

It divides memory into:

• Heap: Stores objects and their instance variables (shared across threads).

• Stack: Each thread has its own stack storing method calls and local variables.

• Method Area (Metaspace in Java 8+): Stores class-level metadata like method definitions and constant pool.

• PC Register: Holds address of current executing instruction.

• Native Method Stack: Used for native (non-Java) code execution.

The JMM ensures visibility, ordering, and atomicity across threads using keywords like volatile, synchronized, and final.

2. What is heap vs stack?

Example:

int x = 10;            // stored in stack
Person p = new Person(); // object p in heap, reference in stack

3. What is PermGen and Metaspace?

• PermGen (Permanent Generation):
  Used before Java 8 to store class metadata, static variables, and interned strings.
  Had a fixed size, which could cause OutOfMemoryError: PermGen space.

• Metaspace (Java 8+):
  Replaced PermGen. It stores class metadata in native memory (not heap).
  Grows dynamically as needed, reducing memory errors.

✅ In short: Metaspace is a more flexible, dynamic replacement for PermGen.

4. What is garbage collection? How does it work?

Garbage Collection (GC) is an automatic memory management process that reclaims memory from objects no longer reachable by any reference.

How it works:

1. JVM identifies unreachable objects (not referenced anywhere).

2. GC frees that memory space.

3. Memory is reused for new objects.

GC Algorithms:

• Serial GC (single-threaded, small apps)

• Parallel GC (multi-threaded)

• G1 GC (Garbage First) – default in Java 9+, low pause time collector.

Phases:

• Mark: Identify live objects.

• Sweep/Compact: Remove dead objects and defragment memory.

Developers can trigger GC via System.gc(), but JVM decides the actual execution time.

5. Finalize method – how and when is it called?

finalize() is a method defined in the Object class:

protected void finalize() throws Throwable

It’s called by the Garbage Collector before reclaiming an object’s memory — a last chance to release resources.

However:

• Execution is not guaranteed or timely.

• It might never be called if the program exits before GC runs.

✅ Since Java 9, finalize() is deprecated.
Modern alternatives:

• Use try-with-resources for closing streams.

• Implement AutoCloseable for cleanup logic.

Access Modifiers

1. Difference between private, protected, public, and default

Java provides four access levels to control visibility of classes, methods, and variables.

Summary:

• private → Most restrictive; used for encapsulation.

• default → Accessible within the same package.

• protected → Visible to subclasses even if they’re in different packages.

• public → Accessible from anywhere.

Example:

public class Person {
    private int age;
    protected String name;
    public void greet() {}
}

2. Can a class be private in Java?

Top-level (outer) classes cannot be private — they must be either public or default.

However, inner classes can be private.

Example:

class Outer {
    private class Inner {   // ✅ allowed
        void show() { System.out.println("Inner"); }
    }
}

If a top-level class were private, no other class (even in the same package) could access it — defeating the purpose of reusability.

3. What are static blocks?

A static block in Java is used for class-level initialization.
It runs once, when the class is first loaded into memory (before any constructor or object creation).

Example:

class DatabaseConnection {
    static {
        System.out.println("Initializing DB connection...");
    }
}

Use cases:

• Initialize static variables.

• Load configurations.

• Perform one-time setup (e.g., registering JDBC drivers).

Order of execution:

1. Static variables.

2. Static blocks (in order of appearance).

3. Constructors (when object is created).

Exception Handling

1. Checked vs Unchecked Exceptions?

Key point:
Checked exceptions represent recoverable errors, while unchecked ones indicate programming bugs (e.g., null access).

2. Difference between throw and throws?

Example:

void readFile() throws IOException {
    throw new IOException("Error reading file");
}

3. Can we have try block without catch/finally?

No ❌
A try block must be followed by at least one of the following:

• A catch block

• A finally block

However, try-finally without catch is valid:

try {
    System.out.println("Try block");
} finally {
    System.out.println("Cleanup code");
}

This ensures cleanup executes even if no exception occurs.

4. What is the use of finally block?

The finally block is used for resource cleanup — it executes always, regardless of whether an exception occurs or not.

Example:

try {
    FileInputStream fis = new FileInputStream("data.txt");
} catch (IOException e) {
    e.printStackTrace();
} finally {
    System.out.println("Closing resources...");
}

✅ Executed:

• After try or catch block

• Even if return is used inside try or catch
  ❌ Not executed if JVM exits via System.exit(0) or power failure.

Modern alternative:
Use try-with-resources (Java 7+) to auto-close resources.

5. Custom exception in Java?

A custom exception allows developers to define domain-specific error types.

Steps to create:

1. Extend Exception (for checked) or RuntimeException (for unchecked).

2. Add constructors for custom messages.

Example:

class InvalidAgeException extends Exception {
    public InvalidAgeException(String msg) {
        super(msg);
    }
}

class Validator {
    void validate(int age) throws InvalidAgeException {
        if (age < 18)
            throw new InvalidAgeException("Age must be >= 18");
    }
}

Use cases:

• Validation (e.g., invalid user input)

• Business logic constraints

• Domain-specific error reporting (e.g., InsufficientBalanceException)

Collections Framework

1. List vs Set vs Map?

Example:

List<String> list = new ArrayList<>();
Set<String> set = new HashSet<>();
Map<Integer, String> map = new HashMap<>();

2. ArrayList vs LinkedList?

Example:

• Use ArrayList when reads are frequent.

• Use LinkedList when insertions/removals are frequent.

3. HashMap vs Hashtable?

✅ For thread-safe alternatives, use ConcurrentHashMap instead of Hashtable.

4. How does HashMap work internally?

HashMap stores data in buckets using a hashing mechanism.

Process:

1. Key’s hashCode() is computed.

2. The hash is mapped to an index in the bucket array ((n - 1) & hash).

3. Each bucket holds a linked list or tree (Java 8+).

4. On put():

   • If key exists → value replaced.

   • If not → new node added.

5. On get():

   • Hash computed → bucket located → key compared using equals().

Optimization (Java 8+):

• If bucket size > 8, converts linked list → balanced tree (Red-Black Tree) for O(log n) lookup.

5. What is the load factor and threshold in HashMap?

• Load Factor: Defines how full the HashMap can get before resizing (default = 0.75).

• Threshold: capacity × loadFactor.
  When the number of entries exceeds this threshold, HashMap resizes (doubles its capacity).

Example:

HashMap<String, Integer> map = new HashMap<>(16, 0.75f);

✅ Resizing improves performance but comes with a cost — it’s best to initialize maps with an estimated size to minimize rehashing.

6. What are fail-fast and fail-safe iterators?

Example:

for (Integer i : list) {
    list.add(10); // ❌ throws ConcurrentModificationException
}

✅ Use fail-safe collections in concurrent environments.

7. What is ConcurrentHashMap and how is it different?

ConcurrentHashMap is a thread-safe alternative to HashMap that provides high concurrency with minimal locking.

How it differs:

• No full-table lock (uses segment-level locking or CAS in Java 8+).

• Null keys/values are not allowed.

• Iterators are fail-safe (operate on snapshot).

• Performs better than Hashtable under multithreading.

Example:

ConcurrentHashMap<String, Integer> map = new ConcurrentHashMap<>();
map.put("A", 1);

8. TreeMap vs HashMap?

Example:

Map<Integer, String> map = new TreeMap<>();
map.put(2, "B");
map.put(1, "A"); // Automatically sorted by key

9. What is LinkedHashMap?

LinkedHashMap maintains insertion order or access order of entries.

Internally:
It extends HashMap and adds a doubly-linked list to preserve order.

Example:

Map<Integer, String> map = new LinkedHashMap<>();
map.put(1, "A");
map.put(2, "B");

✅ Useful when you need predictable iteration order or LRU caching (using removeEldestEntry).

10. When to use ArrayList vs Vector?

✅ Modern Java avoids Vector.
Use ArrayList for single-threaded cases, or Collections.synchronizedList() if synchronization is needed.

Multithreading and Concurrency

1. Difference between process and thread?

In short: Threads share the same heap, enabling lightweight multitasking within a single process.

2. Ways to create a thread in Java?

There are three main ways:

1. Extend Thread class

    class MyThread extends Thread {
        public void run() {
            System.out.println("Thread running...");
        }
    }
    new MyThread().start();
   

2. Implement Runnable interface

    class MyTask implements Runnable {
        public void run() {
            System.out.println("Running via Runnable");
        }
    }
    new Thread(new MyTask()).start();
   

3. Use ExecutorService or Callable (preferred in modern Java)

    ExecutorService service = Executors.newFixedThreadPool(2);
    service.submit(() -> System.out.println("Running in pool"));
   

✅ Best practice: Use ExecutorService to manage threads efficiently.

3. Thread vs Runnable?

Example:
If your class already extends another class, use Runnable since Java doesn’t support multiple inheritance.

4. What is thread lifecycle?

A thread in Java passes through these five states:

1. New – Created but not started (new Thread()).

2. Runnable – Ready to run, waiting for CPU (start() called).

3. Running – Currently executing.

4. Blocked/Waiting – Waiting for a resource or another thread.

5. Terminated – Execution completed or stopped.

Example:

Thread t = new Thread(() -> {});
t.start();  // Runnable

5. Difference between wait, sleep, join?

Example:

Thread t = new Thread(() -> System.out.println("Task"));
t.start();
t.join(); // waits for t to finish

6. What is synchronized block/method?

The synchronized keyword ensures mutual exclusion — only one thread can access a block/method at a time for a given object.

Types:

• Synchronized method

    synchronized void increment() { 
        count++; 
    }
  

• Synchronized block

    void increment() {
        synchronized(this) {
            count++;
        }
    }
  

Benefits: Prevents race conditions.
Downside: Can reduce performance due to locking.

7. What is a deadlock? How to prevent it?

A deadlock occurs when two or more threads are waiting for each other’s locks indefinitely.

Example:

synchronized (obj1) {
    synchronized (obj2) { ... }
}

Another thread might lock obj2 first, then wait for obj1.

Prevention:

• Acquire locks in a consistent order.

• Use tryLock() with timeout (ReentrantLock).

• Avoid nested locks where possible.

8. What is volatile keyword?

The volatile keyword ensures that a variable’s value is always read from main memory, not from a thread’s local cache.

Example:

volatile boolean flag = true;

If one thread changes flag, other threads immediately see the updated value.

Important:

• Ensures visibility, not atomicity.

• For compound operations (e.g., count++), use synchronization or AtomicInteger.

9. Difference between ExecutorService and Thread?

Example:

ExecutorService executor = Executors.newFixedThreadPool(3);
executor.submit(() -> System.out.println("Task executed"));
executor.shutdown();

✅ Best practice: Always use ExecutorService for managing multiple threads efficiently.

10. What is thread safety?

Thread safety means an object or code segment behaves correctly when accessed by multiple threads simultaneously.

Ways to achieve:

• Use immutable objects

• Use synchronized blocks

• Use atomic classes (AtomicInteger, AtomicReference)

• Use concurrent collections (ConcurrentHashMap, CopyOnWriteArrayList)

Example:

AtomicInteger counter = new AtomicInteger();
counter.incrementAndGet();

✅ Thread-safe code avoids race conditions and data inconsistency.

Java 8 Features

1. What are the major features introduced in Java 8?

Java 8 introduced functional programming concepts and several performance improvements.

Key Features:

• Lambda Expressions

• Functional Interfaces

• Stream API

• Optional class

• Method References

• Default and Static methods in Interfaces

• Date and Time API (java.time)

• Parallel Streams

2. What are lambda expressions?

A lambda expression provides a concise way to represent anonymous functions.

Syntax:

(parameter) -> expression

Example:

List<String> names = Arrays.asList("John", "Jane", "Max");
names.forEach(name -> System.out.println(name));

Before Java 8:

for (String name : names)
    System.out.println(name);

✅ Advantages:

• Reduces boilerplate code

• Enables functional programming

• Works well with Streams and Collections

3. What is a functional interface?

A functional interface is an interface that contains exactly one abstract method.
It can have default or static methods as well.

Example:

@FunctionalInterface
interface Calculator {
    int add(int a, int b);
}

Built-in Functional Interfaces (in java.util.function):

• Predicate<T> → returns boolean

• Function<T, R> → transforms T to R

• Consumer<T> → accepts and performs action

• Supplier<T> → returns value without input

Example:

Predicate<Integer> isEven = x -> x % 2 == 0;
System.out.println(isEven.test(4)); // true

4. What is the Stream API?

The Stream API is used to process collections of data in a declarative and functional style.

Example:

List<Integer> nums = Arrays.asList(1, 2, 3, 4, 5);
List<Integer> squares = nums.stream()
                            .map(n -> n * n)
                            .collect(Collectors.toList());
System.out.println(squares);

Key Operations:

• Intermediate → filter(), map(), sorted()

• Terminal → collect(), forEach(), count(), reduce()

✅ Streams don’t modify the original collection.

5. What is the difference between map() and flatMap()?

Example:

List<List<Integer>> list = Arrays.asList(Arrays.asList(1,2), Arrays.asList(3,4));
list.stream().flatMap(Collection::stream).forEach(System.out::println);

6. What is the Optional class?

Optional<T> is a container that may or may not hold a non-null value.
It helps avoid NullPointerException.

Example:

Optional<String> name = Optional.ofNullable(getName());
System.out.println(name.orElse("Unknown"));

Common methods:

• isPresent()

• orElse()

• orElseGet()

• orElseThrow()

• map() and flatMap()

✅ Always use Optional for return types, not fields.

7. What are method references?

Method references provide a shorthand for calling existing methods using ::.

Example:

list.forEach(System.out::println);

Types:

• Static method → ClassName::staticMethod

• Instance method → object::instanceMethod

• Constructor → ClassName::new

Example:

Supplier<List<String>> supplier = ArrayList::new;

8. What are default and static methods in interfaces?

Default methods: Provide a method body inside an interface.

Example:

interface Vehicle {
    default void start() {
        System.out.println("Vehicle started");
    }
}

Static methods in interfaces belong to the interface itself:

interface Utils {
    static void show() {
        System.out.println("Static method in interface");
    }
}

✅ These features were introduced to maintain backward compatibility when new methods were added to interfaces like List and Map.

9. What is the new Date and Time API (java.time)?

Java 8 introduced a modern, immutable, thread-safe Date-Time API.

Key Classes:

• LocalDate, LocalTime, LocalDateTime

• ZonedDateTime

• Period, Duration

• DateTimeFormatter

Example:

LocalDate today = LocalDate.now();
LocalDate tomorrow = today.plusDays(1);
System.out.println(tomorrow);

✅ No more java.util.Date and SimpleDateFormat pain!

10. What are parallel streams?

Parallel streams allow data to be processed in multiple threads automatically.

Example:

list.parallelStream()
    .filter(x -> x > 10)
    .forEach(System.out::println);

Note:

• Use parallel streams for CPU-intensive operations.

• Avoid for IO-bound or small datasets (due to overhead).

Strings

1. String vs StringBuilder vs StringBuffer?

Example:

String s = "Hello";
s.concat(" World"); // New object created

StringBuilder sb = new StringBuilder("Hello");
sb.append(" World"); // Modified in place

✅ For most use cases → prefer StringBuilder (fast, flexible).

2. How are strings stored in memory?

• Strings are stored in a special area of heap memory called the String Constant Pool (SCP).

• When you create a string literal, e.g., "Java", it’s interned — meaning:

  • If "Java" already exists in the pool, the same reference is reused.

  • If not, it’s added to the pool.

Example:

String s1 = "Java";
String s2 = "Java";
System.out.println(s1 == s2); // true (same reference)

✅ String literals are interned;
new String("Java") creates a new object on the heap (not in SCP).

3. What is the String Constant Pool?

The String Constant Pool (SCP) is part of the heap memory (since Java 7+).
It’s a cache that stores unique string literals to improve performance and save memory.

Example:

String a = "Test";
String b = "Test";
System.out.println(a == b); // true (both refer to same object)

If created using new:

String c = new String("Test");
System.out.println(a == c); // false (different objects)

✅ You can manually add strings to the pool using intern():

String d = c.intern();
System.out.println(a == d); // true

4. Why are Strings immutable in Java?

Reasons:

1. Security: Used in sensitive contexts (ClassLoader, File paths, URLs).

2. Thread-safety: Immutable objects can be shared safely between threads.

3. Caching: Hash code of a string is cached; immutability ensures it’s consistent.

4. String Pool Optimization: Same string literals can be reused safely.

Example:

String s = "Java";
s.concat(" Rocks"); // creates a new string

✅ Once created, the value of a String object cannot be changed.

5. How does equals() work in String?

The equals() method compares the contents (values) of two strings,
while the == operator compares references.

Example:

String s1 = new String("Hello");
String s2 = new String("Hello");

System.out.println(s1 == s2);       // false (different objects)
System.out.println(s1.equals(s2));  // true (same content)

✅ String overrides equals() and hashCode() from Object class to compare actual text.

6. How does hashCode() work in Strings?

• The hash code for a string is computed based on its characters:

    s[0]*31^(n-1) + s[1]*31^(n-2) + ... + s[n-1]
  

• Because strings are immutable, their hash code is cached for performance.

✅ Used extensively in collections like HashMap and HashSet.

7. How do substring(), split(), and join() work?

substring(begin, end)

• Returns part of the string.

• Creates a new String (does not modify original).

Example:

String s = "developer";
System.out.println(s.substring(0, 3)); // "dev"

split(regex)

• Splits a string into an array based on a delimiter.

String s = "a,b,c";
String[] arr = s.split(",");

String.join(delimiter, elements...)

String result = String.join("-", "Java", "Python", "C++");
System.out.println(result); // "Java-Python-C++"

8. What is String interning?

Interning ensures that identical strings share the same reference in the String Pool.

Example:

String s1 = new String("Java");
String s2 = s1.intern();
String s3 = "Java";

System.out.println(s2 == s3); // true

✅ Improves memory efficiency, especially when there are many repeated strings.

9. What are common StringBuilder methods?

• append() – concatenates data

• insert() – inserts at position

• delete() – removes substring

• reverse() – reverses content

• capacity() – returns current buffer capacity

Example:

StringBuilder sb = new StringBuilder("Hello");
sb.append(" World").reverse();
System.out.println(sb); // "dlroW olleH"

10. Can we make String mutable in Java?

Not directly.
However, you can simulate mutability using:

• StringBuilder or StringBuffer

• Reflection (not recommended)

• Creating a wrapper class with a mutable reference internally

Java Keywords and Modifiers

1. Difference between final, finally, and finalize?

Summary:

• final → compile-time constant, prevents inheritance/overriding

• finally → runtime block for cleanup

• finalize() → deprecated method, used for cleanup before garbage collection

2. What does static mean?

• Static members belong to the class, not instances.

• Static variable: Shared across all objects

• Static method: Can be called without an object

• Static block: Executes once at class loading

Example:

class Demo {
    static int count;
    static { System.out.println("Class loaded"); }
    static void show() { System.out.println("Static method"); }
}
Demo.show();
System.out.println(Demo.count);

✅ Use static for memory efficiency and shared state.

3. What is transient?

• Transient is a keyword used in serialization.

• Fields marked as transient are not serialized.

Example:

class User implements Serializable {
    private String name;
    private transient String password; // won’t be saved
}

✅ Useful for sensitive data like passwords or temporary fields.

4. What is volatile?

• Ensures visibility of changes across threads.

• Guarantees that reads/writes go directly to main memory.

• Does not guarantee atomicity.

Example:

volatile boolean running = true;

while (running) { 
    /* do work */ 
}

Without volatile, one thread may never see the updated value.

5. Difference between this and super?

• this() → calls current class constructor

• super() → calls parent class constructor

• Both must be the first statement in constructor when used

6. Additional Notes on Keywords/Modifiers

• abstract: Cannot instantiate class, may contain abstract methods

• synchronized: Ensures mutual exclusion for threads

• native: Method implemented in platform-specific code (C/C++)

• strictfp: Ensures floating-point calculations are platform-independent

• default (in interfaces): Provides default method implementation

• var (Java 10+): Type inference for local variables

Java Inner Classes

1. What are inner classes?

An inner class is a class defined within another class.
They allow logical grouping of classes and access to private members of the outer class.

Types of inner classes:

1. Member (non-static) inner class

2. Static nested class

3. Local inner class

4. Anonymous inner class

2. What are static nested classes?

• A static nested class is declared with the static keyword.

• Unlike member inner classes, it does not have access to instance variables of the outer class.

• Can be instantiated without an object of the outer class.

Example:

class Outer {
    static class Nested {
        void show() { System.out.println("Static nested class"); }
    }
}
Outer.Nested nested = new Outer.Nested();
nested.show();

✅ Useful for grouping classes logically and reducing namespace pollution.

3. Difference between local, anonymous, and member classes

Key points:

• Member inner classes hold a reference to outer class.

• Static nested classes don’t hold outer class reference, so memory footprint is smaller.

• Anonymous classes are great for event handlers and callbacks.

Example of anonymous inner class:

Runnable r = new Runnable() {
    @Override
    public void run() {
        System.out.println("Anonymous Runnable");
    }
};
new Thread(r).start();

4. When to use inner classes?

• When a class is only relevant to its outer class.

• For callbacks, listeners, or adapters.

• To encapsulate helper classes without exposing them publicly.

• To access outer class private members without getters/setters.

✅ Correct use of inner classes improves code readability, encapsulation, and design.

Design Principles / Patterns

1. What is SOLID?

SOLID is an acronym for five design principles that make code more maintainable, scalable, and testable:

✅ These principles are key to writing clean and scalable Java applications.

2. What is the Singleton pattern?

The Singleton pattern ensures a class has only one instance and provides a global access point.

Implementation (Thread-safe, lazy initialization):

class Singleton {
    private static volatile Singleton instance;

    private Singleton() {} // private constructor

    public static Singleton getInstance() {
        if (instance == null) {
            synchronized(Singleton.class) {
                if (instance == null) {
                    instance = new Singleton();
                }
            }
        }
        return instance;
    }
}

✅ Use cases:

• Logger

• Configuration manager

• Thread pool manager

3. How to implement an immutable class?

Steps to make a class immutable:

1. Declare class as final.

2. Make all fields private and final.

3. No setters.

4. Return deep copies of mutable objects.

Example:

final class Employee {
    private final String name;
    private final int age;

    public Employee(String name, int age) {
        this.name = name;
        this.age = age;
    }

    public String getName() { return name; }
    public int getAge() { return age; }
}

✅ Immutable objects are thread-safe and prevent unintended state changes.

4. What is the Factory pattern?

The Factory pattern provides a way to create objects without exposing instantiation logic.

Example:

interface Shape { void draw(); }

class Circle implements Shape {
    public void draw() { System.out.println("Circle"); }
}

class Square implements Shape {
    public void draw() { System.out.println("Square"); }
}

class ShapeFactory {
    public static Shape getShape(String type) {
        if (type.equalsIgnoreCase("circle")) return new Circle();
        if (type.equalsIgnoreCase("square")) return new Square();
        throw new IllegalArgumentException("Unknown shape");
    }
}

// Usage
Shape s = ShapeFactory.getShape("circle");
s.draw();

✅ Use when object creation logic is complex or depends on conditions.

5. What is Dependency Injection (DI)?

Dependency Injection is a design pattern where an object receives its dependencies from external sources rather than creating them internally.

Types of DI:

• Constructor injection – dependencies passed via constructor

• Setter injection – dependencies set via setters

• Interface injection – using an interface to inject dependencies (less common)

Example (Constructor DI):

class Service {
    private final Repository repo;
    public Service(Repository repo) { this.repo = repo; }
}
Repository repo = new Repository();
Service service = new Service(repo);

✅ Benefits:

• Promotes loose coupling

• Easier unit testing

• Improves maintainability

JVM Internals and Performance

1. What happens when you run a Java program?

Steps in Java program execution:

1. Compilation: .java source code is compiled into .class bytecode using javac.

2. Class loading: JVM loads the .class files into memory using ClassLoaders.

3. Bytecode verification: Ensures code is valid and doesn’t violate JVM constraints.

4. Execution: Just-In-Time (JIT) compiler converts bytecode into native machine code for performance.

5. Memory allocation: Objects are created in the heap; references stored in stack.

6. Garbage Collection: Unreferenced objects are cleaned automatically.

✅ Key point: JVM allows platform independence by abstracting underlying OS and hardware.

2. Explain class loading process

Class loading is handled by the JVM in three phases:

1. Loading: Loads .class file into memory using ClassLoader.

2. Linking:

   • Verification: Ensures bytecode integrity

   • Preparation: Allocates memory for static variables

   • Resolution: Resolves symbolic references

3. Initialization: Executes static blocks and initializes static fields.

3. What is the role of ClassLoader?

ClassLoader loads Java classes into JVM at runtime.

Types of ClassLoaders:

1. Bootstrap ClassLoader: Loads core Java classes (rt.jar)

2. Extension ClassLoader: Loads JDK extension libraries (lib/ext)

3. Application ClassLoader: Loads classes from classpath (your code)

4. Custom ClassLoader: Can be created for dynamic class loading

✅ ClassLoader ensures lazy loading — classes are loaded only when needed.

4. How does Java achieve platform independence?

• Java code is compiled into bytecode (.class file), not native machine code.

• JVM interprets bytecode or uses JIT to convert to native code.

• Same bytecode can run on any platform with a compatible JVM.

Key principle: Write Once, Run Anywhere (WORA).

5. How to improve Java performance?

• Memory management:

  • Use proper data structures (ArrayList vs LinkedList)

  • Minimize object creation, reuse objects

• Multithreading: Use ExecutorService, parallel streams wisely

• Collections: Use Concurrent collections for thread safety

• Garbage Collection tuning: Adjust heap size and GC algorithm

• String handling: Use StringBuilder instead of String concatenation in loops

• Profiling: Use tools like VisualVM, JConsole to detect bottlenecks

6. Garbage Collection overview

• Automatic memory management by JVM

• Removes unreachable objects from heap

• Common collectors: Serial, Parallel, CMS, G1

• GC events: Minor GC (young generation), Major GC (old generation)

• Finalize() is called before GC (deprecated in Java 9+)

✅ Modern JVM uses generational GC for efficiency.

7. Heap vs Stack

• Stack memory is LIFO, automatically cleaned.

• Heap memory is shared, cleaned by GC.

8. PermGen vs Metaspace

• PermGen (Java 7 and below): Stores class metadata; fixed size → risk of OutOfMemoryError.

• Metaspace (Java 8+): Dynamically resizable; stored in native memory → reduces class loading errors.

Miscellaneous Java Concepts

1. Serialization vs Deserialization

• Serialization: Converts a Java object into a byte stream to save to disk or send over a network.

• Deserialization: Converts the byte stream back into a Java object.

Example:

// Serialization
ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream("data.obj"));
oos.writeObject(employee);

// Deserialization
ObjectInputStream ois = new ObjectInputStream(new FileInputStream("data.obj"));
Employee e = (Employee) ois.readObject();

Notes:

• Use transient keyword for fields that shouldn’t be serialized.

• Serializable classes must implement Serializable.

2. What is a marker interface?

• An interface without methods used to mark a class with special behavior.

• Examples: Serializable, Cloneable, Remote.

Purpose: Provides metadata to JVM or frameworks.

3. What is instanceof keyword?

• Checks if an object is an instance of a specific class or implements an interface.

Example:

String s = "Java";
if (s instanceof String) {
    System.out.println("s is a String");
}

✅ Always returns true for null checks: null instanceof String → false.

4. What is reflection API?

• Reflection allows inspecting and manipulating classes, methods, and fields at runtime.

• Can be used to:

  • Get class info (Class<?> clazz = obj.getClass();)

  • Access private fields/methods

  • Dynamically create instances

Example:

Class<?> clazz = Class.forName("java.util.ArrayList");
Object obj = clazz.getDeclaredConstructor().newInstance();

✅ Useful for frameworks like Spring, Hibernate, and testing tools.

5. What is autoboxing and unboxing?

• Autoboxing: Automatic conversion from primitive to wrapper class

    int x = 10;
    Integer y = x; // autoboxing
  

• Unboxing: Wrapper class → primitive

    Integer a = 20;
    int b = a; // unboxing
  

6. What is enum in Java?

• enum is a special class representing a fixed set of constants.

• Can have fields, methods, and constructors.

Example:

enum Day { MONDAY, TUESDAY, WEDNESDAY }
Day today = Day.MONDAY;

✅ Useful for type safety instead of string constants.

7. Can we override private methods?

• No, private methods are not visible to subclasses.

• They are class-specific and cannot be overridden, but they can be redeclared in a subclass.

8. Can we overload main method?

• Yes, you can overload main:

public static void main(String[] args) { }
public static void main(int[] args) { }

• JVM always calls the String[] main method.

9. Can constructor be private?

• Yes, used in Singleton pattern or factory methods.

• Prevents direct instantiation from outside the class.

10. What are annotations in Java?

• Metadata for classes, methods, fields.

• Examples: @Override, @Deprecated, @FunctionalInterface

• Custom annotation example:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
@interface Test { 
}

11. What is JavaBeans?

• JavaBean: A reusable class following:

  • Private fields

  • Public getters and setters

  • No-arg constructor

  • Serializable

✅ Used for encapsulation and frameworks like JSP, JSF, and Spring.

12. What is var keyword in Java?

• Introduced in Java 10 for local variable type inference.

var name = "Ashutosh"; // inferred as String

• Cannot be used for method parameters, fields, or return type.

13. What are records in Java?

• Introduced in Java 14 as immutable data carriers.

• Automatically generate constructor, getters, equals, hashCode, toString.

record Employee(String name, int age) { }
Employee e = new Employee("John", 25);

✅ Great for DTOs and value objects.

Learn more about records here.

14. Difference between compile-time and runtime errors

Real-World Scenario-Based Questions

1. How would you optimize a large collection for search?

Scenario: You have a large dataset and need frequent lookups.

Approach:

1. Choose the right data structure:

   • HashMap for O(1) key-value lookup.

   • TreeMap if sorted order is required (O(log n) lookup).

   • HashSet for unique element search.

2. Indexing: Precompute indices for frequent queries.

3. Use streams with parallel processing if data is very large.

4. Avoid unnecessary object creation: Use primitives or immutable objects.

5. Memory considerations: If memory is tight, consider using compressed data structures or disk-based solutions.

Example:

Map<String, Employee> employeeMap = new HashMap<>();
for(Employee e: employees) {
    employeeMap.put(e.getId(), e);
}
// Lookup by ID is now O(1)

2. How to handle concurrency in a shared resource?

Scenario: Multiple threads need to access/update shared data.

Approach:

1. Synchronized blocks/methods:

    synchronized(this) {
        // critical section
    }
   

2. Concurrent Collections: Use ConcurrentHashMap, CopyOnWriteArrayList.

3. Locks: Use ReentrantLock for fine-grained control.

4. Atomic Variables: Use AtomicInteger, AtomicReference for atomic operations.

5. Avoid deadlocks: Always acquire locks in consistent order.

✅ Modern Java encourages lock-free structures and immutability where possible.

3. What’s your approach for debugging memory leaks?

Steps:

1. Identify symptoms: High heap usage, OutOfMemoryError, slow performance.

2. Analyze heap dumps: Use tools like VisualVM, Eclipse MAT, JProfiler.

3. Check references: Look for unreleased objects, static collections, caches.

4. Fix common causes:

   • Remove unused listeners or callbacks

   • Use WeakReference where needed

   • Clear collections properly

5. Test thoroughly: Run under load to confirm memory usage stabilizes.

4. How to ensure thread safety without performance issues?

Scenario: High-concurrency application.

Approach:

1. Prefer immutable objects to avoid synchronization overhead.

2. Use concurrent collections (ConcurrentHashMap, ConcurrentLinkedQueue) instead of synchronized versions.

3. Minimize synchronized blocks to only critical sections.

4. Use atomic variables for counters or flags.

5. Consider ReadWriteLock if reads dominate writes.

✅ Key: Balance thread safety vs performance.

5. Explain a situation where you used Java 8 features to simplify code

Example Answer:

“In a recent project, we had a list of employees and needed to filter those in a specific department, sort by salary, and collect names. Using Java 8 Streams and lambdas, I could do it in one readable statement instead of nested loops:

List<String> names = employees.stream()
    .filter(e -> e.getDepartment().equals("Engineering"))
    .sorted(Comparator.comparing(Employee::getSalary).reversed())
    .map(Employee::getName)
    .collect(Collectors.toList());

This approach reduced boilerplate code, improved readability, and was easy to maintain.”

Wrapping Up

This guide covers all essential Core Java concepts, coding patterns, and real-world scenarios needed to excel in Java interviews in 2025. Use it to strengthen fundamentals, improve problem-solving, and confidently tackle interviews.

Now, Java introduced a new feature in Java 14, called Records. These are special types of classes that focus on holding data, just like DTOs. The big difference is that Records do a lot of the repetitive work for us. For example, they automatically create methods to get the data (like getters), and they handle equality checks, toString(), and more. This feature became fully available in Java 16, making Records a modern, clean way to work with data in Java.

So, why are we comparing DTOs and Records? Because they both serve a similar purpose — carrying data. However, understanding when to use one over the other is important as Java continues to evolve. In this article, we’ll explore the differences and help you decide which one fits your needs better, especially if you’re working on modern Java applications.

What is a DTO?

A Data Transfer Object (DTO) is a simple Java object that is used to move data between different parts of an application. Think of it as a container for carrying data between layers of your application. For example, in a web application, a DTO might be used to transfer data from the service layer to the controller, or from the controller to the view layer.

DTOs help keep the different parts of an application separated, making the code more organized and easier to maintain. They typically don’t have any business logic or complex behavior. Instead, they just hold data.

How are DTOs implemented?

DTOs are usually implemented as regular Java classes. A typical DTO includes:

• Private fields for the data it holds.

• Getters and Setters to access and modify the data.

• A Constructor to create the object.

• Override methods like toString(), hashCode(), and equals() for comparing and printing the object in a meaningful way.

Here’s an example of a UserDTO class :

import java.util.Objects;

public class UserDTO {
    private String name;
    private int age;
    private String email;

    // Constructor
    public UserDTO(String name, int age, String email) {
        this.name = name;
        this.age = age;
        this.email = email;
    }

    // Getters and Setters
    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public String getEmail() {
        return email;
    }

    public void setEmail(String email) {
        this.email = email;
    }

    // Overriding toString method for meaningful output
    @Override
    public String toString() {
        return "UserDTO{" +
                "name='" + name + '\'' +
                ", age=" + age +
                ", email='" + email + '\'' +
                '}';
    }

    // Overriding equals method for object comparison
    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        UserDTO userDTO = (UserDTO) o;
        return age == userDTO.age && Objects.equals(name, userDTO.name) && Objects.equals(email, userDTO.email);
    }

    // Overriding hashCode method
    @Override
    public int hashCode() {
        return Objects.hash(name, age, email);
    }
}

This UserDTO class holds information about a user: their name, age, and email. It also provides basic functionality like comparing two UserDTO objects (using equals()), generating a unique hash code (using hashCode()), and a toString() method for readable output.

With tools like Lombok, you can avoid manually writing boilerplate code while still having a fully functional DTO. However, with Records, as we’ll explore later, Java offers an alternative that also eliminates much of the boilerplate but with a different design philosophy (immutability by default).

Here’s how you can use the UserDTO:

public class UserDTOUsageExample {
    public static void main(String[] args) {
        UserDTO user = new UserDTO("Ashutosh", 25, "ashutosh@example.com");

        // Access data
        System.out.println(user.getName());
        System.out.println(user.getAge());
        System.out.println(user.getEmail());

        // Using the toString() method
        System.out.println(user);

        // Comparing records
        UserDTO anotherUser = new UserDTO("Vishakha", 22, "vishakha@example.com");
        System.out.println(user.equals(anotherUser));
    }
}

Output:

Ashutosh
25
ashutosh@example.com
UserDTO{name='Ashutosh', age=25, email='ashutosh@example.com'}
false

What is a Java Record?

Java Records are a special type of class introduced in Java 14 (as a preview feature) and fully released in Java 16. They simplify the creation of immutable data carriers. Records are designed to hold data in a concise and readable way, and they eliminate much of the boilerplate code that traditional classes, like DTOs, require.

In a traditional DTO, you manually write constructors, getters, equals(), hashCode(), and toString() methods (as we did earlier). With Records, Java generates all of these for you automatically. This makes them ideal for simple, immutable objects whose main job is to carry data.

Key Features of Java Records

• Immutable by default: Once you create a record, you can't change its data (unlike DTOs, which are typically mutable).

• Compact syntax: You declare the fields and Java generates the constructor, getters, equals(), hashCode(), and toString() automatically.

• No setters: Since records are immutable, they don’t provide setters.

Let’s create a UserRecord that represents the same user data as our earlier UserDTO, but using a record:

public record UserRecord(String name, int age, String email) {
}

That's it! With just one line, Java generates:

• A constructor: new UserRecord(String name, int age, String email)

• Getters for each field: name(), age(), and email()

• An equals() method for comparing two UserRecord objects.

• A hashCode() method to generate a unique hash code.

• A toString() method that returns a string representation like this: UserRecord[name=Ashutosh, age=25, email=ashutosh@example.com].

Here’s how you can use the UserRecord:

public class UserRecordUsageExample {
    public static void main(String[] args) {
        UserRecord user = new UserRecord("Ashutosh", 25, "ashutosh@example.com");

        // Access data
        System.out.println(user.name());
        System.out.println(user.age());
        System.out.println(user.email());

        // Using the toString() method
        System.out.println(user);

        // Comparing records
        UserRecord anotherUser = new UserRecord("Vishakha", 22, "vishakha@example.com");
        System.out.println(user.equals(anotherUser));
    }
}

Output:

Ashutosh
25
ashutosh@example.com
UserRecord[name=Ashutosh, age=25, email=ashutosh@example.com]
false

With UserRecord, we avoided writing getters, constructors, equals(), hashCode(), and toString() manually. Java Records offer a clean, concise way to create immutable objects that only carry data.

Why Use Records?

• Less Boilerplate: You don’t have to write repetitive code like getters or equals() methods.

• Immutable by Design: Ensures the data can't be changed after the object is created, making it safer to use in multi-threaded environments.

• Clear Intent: Using a Record clearly communicates that the object is just for carrying data, without additional behavior or logic.

Comparing DTO and Record

Now that we know about DTO and Records, let’s compare them in this section.

Immutability

Records are immutable by design, meaning once you create a record instance, you can’t change its data. This immutability ensures that the data remains consistent and thread-safe without needing any extra code. For example, in a UserRecord, the fields name, age, and email can be set only when the record is created, and they can't be modified afterward.

On the other hand, DTOs are typically mutable, meaning their fields can be changed after the object is created. To make DTOs immutable, you would have to explicitly avoid setters or design them carefully (e.g., using final fields). Here’s how immutability looks with a record versus a traditional DTO:

• Record: Immutable by default.

• DTO: Requires manual enforcement for immutability, which can lead to more complex code and potential bugs.

public class ImmutabilityExample {
    public static void main(String[] args) {
        UserDTO userDTO = new UserDTO("Ashutosh", 25, "ashutosh@example.com");
        userDTO.setAge(26);  // DTO allows this by default.

        UserRecord userRecord = new UserRecord("Ashutosh", 25, "ashutosh@example.com");
        userRecord.name = "Jane"; // This would result in a compile-time error.
    }
}

Boilerplate Code

One of the biggest advantages of Records is that they significantly reduce boilerplate code. When using a DTO, you often have to manually write getters, setters, constructors, equals(), hashCode(), and toString() methods. With Records, all of this is generated for you automatically.

In contrast, traditional DTOs require more manual coding. Although tools like Lombok can help reduce the amount of boilerplate, they still don’t provide the same level of simplicity as Records. Here’s a comparison:

• Record: Automatically generates constructor, getters, equals(), hashCode(), and toString().

• DTO: Requires manual implementation or the use of tools like Lombok.

Data Representation

Records provide a compact and concise way of representing data. Since the declaration of a Record contains only the fields, the code is cleaner and easier to read. This makes it easier to maintain, especially in projects with a lot of data models.

For example:

// Record: Clean and simple
public record UserRecord(String name, int age, String email) {}

Compare this to a DTO, which typically has a lot more code:

// DTO: More verbose
public class UserDTO {
    private String name;
    private int age;
    private String email;

    // Constructor, Getters, Setters, toString, equals, hashCode...
}

With Records, the intent is clearer: it’s just a data carrier with no extra behavior, whereas DTOs can easily become cluttered with boilerplate or additional logic.

Customization

One area where DTOs have an advantage is in customization. DTOs allow you to add custom logic, such as data validation, transformation methods, or even business logic if needed (although this is generally discouraged in pure DTOs). For example, you could add a validation method to ensure the email field follows a valid format.

With Records, customization is more limited. Since they are designed to be lightweight and immutable, you can’t easily add custom methods that modify internal state or perform complex logic. If your use case requires custom behavior or logic in your data objects, DTOs offer more flexibility.

Here’s a quick example of adding custom validation to a DTO:

public class UserDTO {
    private String email;

    // Method to validate email format
    public boolean isValidEmail() {
        return email != null && email.contains("@");
    }
}

With a Record, this level of customization would typically be handled outside the Record itself. Records focus strictly on carrying data, while logic like validation is expected to be handled elsewhere.

Alignment with Functional Programming

One of the key principles of functional programming is immutability — the idea that data objects should not be changed after they are created. Records align more closely with functional programming principles because they are immutable by default. This makes them an ideal choice in systems that favor or adopt a functional programming style.

• Records:

  • Designed to be immutable, which aligns with functional programming's emphasis on creating data structures that cannot change state.

  • They promote a more declarative style, where you can pass around immutable data objects without side effects, making them predictable and easier to reason about.

In contrast, DTOs are traditionally mutable by nature. While it’s possible to make DTOs immutable (by avoiding setters and using final fields), it requires manual enforcement. DTOs often follow the object-oriented paradigm, where state changes are more common.

• DTOs:

  • More flexible in terms of mutability, which makes them better suited to imperative or object-oriented programming styles.

  • When used in their mutable form, DTOs can lead to side effects, which is generally discouraged in functional programming.

When to Use DTO vs Record

When deciding between using a DTO or a Record, the choice depends largely on your specific use case, project requirements, and the version of Java you're using. Below is a breakdown of when to use each:

When to Use DTOs

1. When mutability is required:
   If your object’s data needs to be modified after creation, DTOs are the better choice. DTOs are typically mutable, allowing you to change the values of fields as needed. This is useful in scenarios where data is updated throughout the lifecycle of an object.

   Example: In a web application, a form submission may initially create a UserDTO with some fields left blank. As the user updates their profile, the UserDTO may need to change accordingly.

2. When additional behavior or validation logic is needed:
   DTOs are more flexible when it comes to adding custom behavior like validation, transformations, or additional methods. If your data object needs logic beyond simply carrying data (e.g., verifying an email format or sanitizing input), then a DTO is more suitable.

   Example: Adding a method in UserDTO to validate the format of an email before passing it between layers of your application.

3. Compatibility with older versions of Java (pre-Java 16):
   If your project is running on a version of Java earlier than Java 16, you won’t be able to use Records. In these cases, you’ll need to use traditional DTOs or alternatives like Lombok to simplify the code.

   Example: If your application must support Java 11 or Java 8, then Records are not an option, and you’ll stick with DTOs.

When to Use Records

1. When you need a concise, immutable data carrier:
   Records are ideal when you need a lightweight, immutable object to carry data. Since they automatically generate essential methods (constructor, getters, equals(), hashCode(), and toString()), they offer a clean and efficient way to represent data.

   Example: If you’re transferring data between services in a microservice architecture and don't need to modify the data, a UserRecord would be a perfect fit.

2. For read-only data transfer between layers or services:
   If your application involves passing data around without the need to modify it, using a Record is a great choice. The immutability of Records ensures that the data remains consistent, making it suitable for cases like sending data from the database to a service layer or from one service to another.

   Example: A Record might be used to send user data from a database layer to a REST controller in a web application.

3. In modern Java applications (Java 16+):
   If your project uses Java 16 or later, you can take full advantage of Records. They are designed to simplify data representation in modern Java applications and help reduce the boilerplate that comes with traditional DTOs.

   Example: In a Java 17 web service, you might use Records for all your data transfer needs between different layers of your application to keep the codebase concise and maintainable.

Performance Considerations

When comparing DTOs and Records in terms of performance, the differences are typically minimal, but there are a few important factors to consider:

Memory Efficiency

Since Records are compact by design, they may consume slightly less memory than traditional DTOs. The key reason is that Records do not require the additional overhead of manually implementing getters, setters, equals(), hashCode(), and toString() methods. All of this is generated automatically by the Java compiler in a more optimized way, resulting in a smaller memory footprint.

For example:

• A DTO would need separate fields and methods for each operation (getName(), setName(), etc.).

• A Record internally holds just the fields and automatically generates the necessary methods, potentially using fewer resources.

Immutability and Thread-Safety

The immutable nature of Records provides some inherent performance benefits, particularly in multi-threaded environments. Since Records are immutable, they don’t require synchronization or locking mechanisms when shared between threads. This can lead to better performance in scenarios where thread contention would normally degrade performance.

In contrast, if you use mutable DTOs in multi-threaded environments, you need to ensure thread safety, either by synchronizing access or using other mechanisms, which can introduce overhead and slow down the application.

Garbage Collection

Both DTOs and Records are plain Java objects (POJOs), so they are subject to the same garbage collection process. However, the concise nature of Records could lead to slightly faster garbage collection, as fewer objects are created or held in memory. This can contribute to improved performance in long-running applications or those handling large volumes of data objects.

CPU Overhead

Since Records are auto-generated by the compiler and are optimized for performance, there may be slight CPU performance improvements in operations such as object creation, method invocation, and comparison (equals(), hashCode()). This is particularly true when comparing complex DTOs where developers might introduce inefficiencies in manually implemented methods. The uniformity and optimization of Records ensure that these operations are handled consistently and efficiently.

Real-World Performance Impact

In practice, the performance differences between DTOs and Records will likely be small and often negligible for most applications. The compact nature of Records might lead to slight performance gains in some scenarios, but the actual impact would only be noticeable in applications with heavy data processing, high throughput, or those running in resource-constrained environments (e.g., mobile or IoT devices).

Wrapping Up

In this tutorial, we've explored the key differences between DTOs and Records, their respective use cases, and how they align with different programming paradigms like functional programming. While DTOs offer flexibility, mutability, and custom behavior, Records provide a concise and immutable way to model data, making them ideal for modern Java applications.

The decision to use a DTO or a Record ultimately depends on your specific requirements:

• If you need mutability or want to add custom logic, DTOs are a better fit.

• If you prefer a compact, immutable structure and are working in Java 16 or later, Records offer a cleaner and more efficient option.

Both approaches have their strengths, and understanding when to use each will help you write more efficient and maintainable Java code.

Why Use a Table of Contents?

• Enhanced Navigation: TOCs allow readers to quickly jump to sections of interest.

• Improved Readability: A clear structure helps understand the document's flow.

• Better SEO: Well-organized content is more likely to rank higher in search engines.

Key Features of the Application

• Supports freeCodeCamp Draft Posts: Easily generate TOCs for your drafts using the platform's unique URL.

• Markdown Support: Quickly convert your Markdown headers into a clickable TOC.

• User-Friendly Interface: Simple and intuitive design for a seamless experience.

Here's a demo of the application:

How to Use the TOC Generator for freeCodeCamp

1. Access the Application

   • Visit the TOC Generator website.

2. Select the Level

   • Choose the "Single Level" or "Multi-Level" as per your choice.

3. Enter the Preview URL

   • Input your Hashnode Preview URL. The placeholder will show an example URL format for guidance: https://preview.freecodecamp.org/66bf202fb513336e7843a932.
     Here's how you can get the preview URL:

     

4. Generate TOC

   • Click the "Generate TOC" button to create a structured TOC.

You can copy the generated TOC and paste it into your article wherever you want. Hashnode will automatically convert the Markdown to proper links.

Sample TOC with Single Level Headings:

How to Use the TOC Generator for Markdown

1. Access the Markdown TOC Generator

   • Visit our Markdown TOC Generator page.

2. Enter Your Markdown Content

   • Paste or type your Markdown content into the provided textbox.

3. Generate TOC

   • Click "Generate TOC" to automatically generate the TOC based on your headers.

4. Copy the TOC

   • Use the "Copy" button to copy the TOC to your clipboard.

Benefits of Using Our Application

• Time-Saving: Quickly generate TOCs without manual formatting.

• Consistency: Ensure your TOC is always formatted correctly.

• Versatility: Suitable for various platforms and content types.

Conclusion

My Table of Contents Generator is designed to streamline the process of organizing your content, making it easier for your readers to navigate and understand. Whether you're a freeCodeCamp contributor or a Markdown enthusiast, this tool can enhance your workflow and improve the presentation of your work. Try it out today and see how it can simplify your content creation process!

However, when it comes to sorting custom objects, such as instances of user-defined classes, the built-in sorting methods fall short. These methods don't know how to order objects based on custom criteria. This is where Java's Comparable and Comparator interfaces come into play, allowing developers to define and implement custom sorting logic tailored to specific requirements.

In this blog post, we'll explore how to use the Comparable and Comparator interfaces to sort custom objects in Java. I'll provide examples to illustrate the differences and use cases for each approach, helping you master custom sorting in your Java applications.

Sorting Methods for Primitive Types

Java provides a variety of built-in sorting methods that make it easy to sort primitive data types. These methods are highly optimized and efficient, allowing you to sort arrays and collections with minimal code. For primitive types, such as integers, floating-point numbers, and characters, the Arrays.sort() method is commonly used.

Arrays.sort()

The Arrays.sort() method sorts the specified array into ascending numerical order. This method uses a Dual-Pivot Quicksort algorithm, which is faster and more efficient for most data sets.

Let's look at an example of sorting an array of integers and characters using Arrays.sort():

package tutorial;

import java.util.Arrays;

public class PrimitiveSorting {
    public static void main(String[] args) {
        int[] numbers = { 5, 3, 8, 2, 1 };
        System.out.println("Original array: " + Arrays.toString(numbers));

        Arrays.sort(numbers);
        System.out.println("Sorted array: " + Arrays.toString(numbers));

        char[] characters = { 'o', 'i', 'e', 'u', 'a' };
        System.out.println("Original array: " + Arrays.toString(characters));

        Arrays.sort(characters);
        System.out.println("Sorted array: " + Arrays.toString(characters));
    }
}

Output:

Original array: [5, 3, 8, 2, 1]
Sorted array: [1, 2, 3, 5, 8]
Original array: [o, i, e, u, a]
Sorted array: [a, e, i, o, u]

Collections.sort()

The Collections.sort() method is used to sort collections such as ArrayList. This method is also based on the natural ordering of the elements or a custom comparator.

package tutorial;

import java.util.ArrayList;
import java.util.Collections;

public class CollectionsSorting {
    public static void main(String[] args) {
        ArrayList<String> wordsList = new ArrayList<>();
        wordsList.add("banana");
        wordsList.add("apple");
        wordsList.add("cherry");
        wordsList.add("date");
        System.out.println("Original list: " + wordsList);

        Collections.sort(wordsList);
        System.out.println("Sorted list: " + wordsList);
    }
}

Output:

Original list: [banana, apple, cherry, date]
Sorted list: [apple, banana, cherry, date]

Limitations with Custom Classes

While Java's built-in sorting methods, such as Arrays.sort() and Collections.sort(), are powerful and efficient for sorting primitive types and objects with natural ordering (like String), they fall short when it comes to sorting custom objects. These methods do not inherently know how to order user-defined objects because there is no natural way for them to compare these objects.

For example, consider a simple Person class that has name, age, and weight attributes:

package tutorial;

public class Person {
    String name;
    int age;
    double weight;

    public Person(String name, int age, double weight) {
        this.name = name;
        this.age = age;
        this.weight = weight;
    }

    @Override
    public String toString() {
        return "Person [name=" + name + ", age=" + age + ", weight=" + weight + " kgs]";
    }
}

If we try to sort a list of Person objects using Arrays.sort() or Collections.sort(), we will encounter a compilation error because these methods do not know how to compare Person objects:

package tutorial;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

public class CustomClassSorting {
    public static void main(String[] args) {
        List<Person> people = new ArrayList<>(Arrays.asList(
                new Person("Alice", 30, 65.5),
                new Person("Bob", 25, 75.0),
                new Person("Charlie", 35, 80.0)
        ));
        System.out.println("Original people list: " + people);

        Collections.sort(people);
        System.out.println("Sorted people list: " + people);
    }
}

Compilation Error:

java: no suitable method found for sort(java.util.List<tutorial.Person>)
    method java.util.Collections.<T>sort(java.util.List<T>) is not applicable
      (inference variable T has incompatible bounds
        equality constraints: tutorial.Person
        lower bounds: java.lang.Comparable<? super T>)
    method java.util.Collections.<T>sort(java.util.List<T>,java.util.Comparator<? super T>) is not applicable
      (cannot infer type-variable(s) T
        (actual and formal argument lists differ in length))

The error occurs because the Person class does not implement the Comparable interface, and there is no way for the sorting method to know how to compare two Person objects.

To sort custom objects like Person, we need to provide a way to compare these objects. Java offers two main approaches to achieve this:

1. Implementing the Comparable Interface: This allows a class to define its natural ordering by implementing the compareTo method.

2. Using the Comparator Interface: This allows us to create separate classes or lambda expressions to define multiple ways of comparing objects.

We will explore both approaches in the upcoming sections, starting with the Comparable interface.

Comparable Interface

Java provides a Comparable interface to define a natural ordering for objects of a user-defined class. By implementing the Comparable interface, a class can provide a single natural ordering that can be used to sort its instances. This is particularly useful when you need a default way to compare and sort objects.

Overview

The Comparable interface contains a single method, compareTo(), which compares the current object with the specified object for order. The method returns:

• A negative integer if the current object is less than the specified object.

• Zero if the current object is equal to the specified object.

• A positive integer if the current object is greater than the specified object.

How Comparable Allows for a Single Natural Ordering of Objects

By implementing the Comparable interface, a class can ensure that its objects have a natural ordering. This allows the objects to be sorted using methods like Arrays.sort() or Collections.sort() without the need for a separate comparator.

Let's implement the Comparable interface in a new PersonV2 class, comparing by age.

package tutorial;

public class PersonV2 implements Comparable<PersonV2> {
    String name;
    int age;
    double weight;

    public PersonV2(String name, int age, double weight) {
        this.name = name;
        this.age = age;
        this.weight = weight;
    }

    @Override
    public String toString() {
        return "PersonV2 [name=" + name + ", age=" + age + ", weight=" + weight + " kgs]";
    }

    @Override
    public int compareTo(PersonV2 other) {
        return this.age - other.age;
    }
}

In this implementation, the compareTo() method compares the age attribute of the current PersonV2 object with the age attribute of the specified PersonV2 object by subtracting one age from the other. By using the expression this.age - other.age, we’re effectively implementing this logic as follows:

• If this.age is less than other.age, the result will be negative.

• If this.age is equal to other.age, the result will be zero.

• If this.age is greater than other.age, the result will be positive.

Note: We can also use Integer.compare(this.age, other.age) instead of performing the arithmetic operation manually.

Now that the PersonV2 class implements the Comparable interface, we can sort a list of PersonV2 objects using Collections.sort():

package tutorial;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

public class CustomClassSortingV2 {
    public static void main(String[] args) {
        List<PersonV2> people = new ArrayList<>(Arrays.asList(
                new PersonV2("Alice", 30, 65.5),
                new PersonV2("Bob", 25, 75.0),
                new PersonV2("Charlie", 35, 80.0)
        ));
        System.out.println("Original people list: " + people);

        Collections.sort(people);
        System.out.println("Sorted people list: " + people);
    }
}

Output:

Original people list: [PersonV2 [name=Alice, age=30, weight=65.5 kgs], PersonV2 [name=Bob, age=25, weight=75.0 kgs], PersonV2 [name=Charlie, age=35, weight=80.0 kgs]]
Sorted people list: [PersonV2 [name=Bob, age=25, weight=75.0 kgs], PersonV2 [name=Alice, age=30, weight=65.5 kgs], PersonV2 [name=Charlie, age=35, weight=80.0 kgs]]

In this example, the PersonV2 objects are sorted in ascending order of age using the Collections.sort() method, which relies on the natural ordering defined by the compareTo() method in the PersonV2 class.

Limitations of Comparable

While the Comparable interface provides a way to define a natural ordering for objects, it has several limitations that can restrict its use in practical applications. Understanding these limitations can help us determine when to use other mechanisms, such as the Comparator interface, to achieve more flexible sorting.

• Single Natural Ordering: The primary limitation of Comparable is that it allows only one natural ordering for the objects of a class. When you implement Comparable, you define a single way to compare objects, which is used whenever the objects are sorted or compared. This can be restrictive if you need to sort objects in multiple ways.

• Inflexibility: If you need to sort objects by different attributes or in different orders, you will have to modify the class or create new implementations of Comparable. This inflexibility can lead to a proliferation of comparison methods and can make the code harder to maintain.

• Non-Adaptable: Once a class implements Comparable, the natural ordering is fixed and cannot be easily changed. For instance, if your PersonV2 class initially sorts by age but later you need to sort by weight or name, you have to either change the compareTo() method or create a new version of the class.

This is where the Comparator interface comes into play. To define multiple ways of comparing objects, we can use the Comparator interface, which we will explore in the next section.

Comparator Interface

The Comparator interface in Java provides a way to define multiple ways to compare and sort objects. Unlike the Comparable interface, which allows only a single natural ordering, Comparator is designed to offer flexibility by allowing multiple sorting strategies. This makes it particularly useful for scenarios where objects need to be sorted in different ways.

Overview

The Comparator interface defines a single method, compare(), which compares two objects and returns:

• A negative integer if the first object is less than the second object.

• Zero if the first object is equal to the second object.

• A positive integer if the first object is greater than the second object.

This method provides a way to define custom ordering for objects without modifying the class itself.

How Comparator Allows for Multiple Ways of Ordering Objects

The Comparator interface allows you to create multiple Comparator instances, each defining a different ordering for objects. This flexibility means you can sort objects by various attributes or in different orders without altering the object's class.

Let's implement multiple Comparator instances for the Person class. We'll define comparators for sorting by name, by age, and by weight. First, we need to update the Person class to include getters and ensure that attributes are accessible.

package tutorial;

public class Person {
    String name;
    int age;
    double weight;

    public Person(String name, int age, double weight) {
        this.name = name;
        this.age = age;
        this.weight = weight;
    }

    public String getName() {
        return name;
    }

    public int getAge() {
        return age;
    }

    public double getWeight() {
        return weight;
    }

    @Override
    public String toString() {
        return "Person [name=" + name + ", age=" + age + ", weight=" + weight + " kgs]";
    }
}

Comparator by Name

This comparator sorts Person objects alphabetically by their name.

package tutorial.comparator;

import tutorial.Person;

import java.util.Comparator;

public class PersonNameComparator implements Comparator<Person> {

    @Override
    public int compare(Person p1, Person p2) {
        return p1.getName().compareTo(p2.getName());
    }
}

Comparator by Age

This comparator sorts Person objects by their age, in ascending order.

package tutorial.comparator;

import tutorial.Person;

import java.util.Comparator;

public class PersonAgeComparator implements Comparator<Person> {

    @Override
    public int compare(Person p1, Person p2) {
        return p1.getAge() - p2.getAge();
    }
}

Comparator by Weight

This comparator sorts Person objects by their weight, in ascending order.

package tutorial.comparator;

import tutorial.Person;

import java.util.Comparator;

public class PersonWeightComparator implements Comparator<Person> {

    @Override
    public int compare(Person p1, Person p2) {
        return (int) (p1.getWeight() - p2.getWeight());
    }
}

Now, here’s how you can use these Comparator instances to sort a list of Person objects:

package tutorial;

import tutorial.comparator.PersonAgeComparator;
import tutorial.comparator.PersonNameComparator;
import tutorial.comparator.PersonWeightComparator;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

public class CustomClassSortingV3 {
    public static void main(String[] args) {
        List<Person> people = new ArrayList<>(Arrays.asList(
                new Person("Alice", 30, 65.5),
                new Person("Bob", 25, 75.0),
                new Person("Charlie", 35, 80.0)
        ));
        System.out.println("Original people list: " + people);

        Collections.sort(people, new PersonNameComparator());
        System.out.println("Sorted people list by name: " + people);

        Collections.sort(people, new PersonAgeComparator());
        System.out.println("Sorted people list by age: " + people);

        Collections.sort(people, new PersonWeightComparator());
        System.out.println("Sorted people list by weight: " + people);
    }
}

Output:

Original people list: [Person [name=Alice, age=30, weight=65.5 kgs], Person [name=Bob, age=25, weight=75.0 kgs], Person [name=Charlie, age=35, weight=80.0 kgs]]
Sorted people list by name: [Person [name=Alice, age=30, weight=65.5 kgs], Person [name=Bob, age=25, weight=75.0 kgs], Person [name=Charlie, age=35, weight=80.0 kgs]]
Sorted people list by age: [Person [name=Bob, age=25, weight=75.0 kgs], Person [name=Alice, age=30, weight=65.5 kgs], Person [name=Charlie, age=35, weight=80.0 kgs]]
Sorted people list by weight: [Person [name=Alice, age=30, weight=65.5 kgs], Person [name=Bob, age=25, weight=75.0 kgs], Person [name=Charlie, age=35, weight=80.0 kgs]]

In this example, the Comparator instances allow sorting the Person objects by different attributes: name, age, and weight. This demonstrates how the Comparator interface enables flexible and versatile sorting strategies for a class.

Comparable vs Comparator

When sorting objects in Java, you have two primary options: the Comparable and Comparator interfaces. Understanding the differences between these two interfaces can help you choose the right approach for your needs. Please note that this is also a very important interview question.

Comparison

Here’s a table comparing and contrasting the Comparable and Comparator interfaces in Java:

Benefits and Drawbacks of Each Approach

Comparable

• Benefits:

  • Simplicity: Provides a default sorting order that is easy to implement and use.

  • Built-in: The natural ordering is part of the class itself, so it is always available and used by default in sorting methods.

• Drawbacks:

  • Single Ordering: Can only define one way to compare objects. If different sorting orders are needed, the class must be modified or additional Comparator instances must be used.

  • Class Modification: Requires altering the class to implement Comparable, which might not be feasible if the class is part of a library or if its natural ordering is not clear.

Comparator

• Benefits:

  • Flexibility: Allows for multiple sorting orders and criteria, which can be defined externally and used as needed.

  • Non-invasive: Does not require modification of the class itself, making it suitable for classes you do not control or when you need different sorting options.

• Drawbacks:

  • Complexity: Requires creating and managing multiple Comparator instances, which can add complexity to the code.

  • Overhead: Might introduce additional overhead if many comparators are used, especially if they are created on the fly.

In summary, Comparable is best used when a class has a natural ordering that makes sense for most use cases. Comparator, on the other hand, provides flexibility for sorting by multiple criteria and is useful when the class does not have a natural ordering or when different sorting orders are needed. Choosing between Comparable and Comparator depends on your specific sorting needs and whether you need a single default order or multiple flexible sorting options.

Wrapping Up

Understanding and utilizing both Comparable and Comparator can significantly enhance your ability to manage and manipulate object collections in Java. By applying these concepts, you can create more flexible and powerful sorting mechanisms.

To solidify your understanding, try implementing both Comparable and Comparator in real-world scenarios. Experiment with different classes and sorting criteria to see how each approach works in practice.

Links to Official Java Documentation:

• Java Comparable Interface

• Java Comparator Interface

Instant search is a feature that shows search results as users type their query. Instead of waiting for a full page to reload or submitting a form, results appear instantly, allowing users to find what they are looking for more quickly. For example, when you start typing in a search box, suggestions or matching items will appear immediately, making the process smoother and more efficient.

In this tutorial, we will learn how to create a simple instant search feature using Flask and HTMX. This will help you build interactive web applications with a better user experience.

Why Use Instant Search?

• Speed: Users get immediate feedback, which helps them refine their search.

• Convenience: It reduces the number of clicks and page loads, leading to a more seamless experience.

• Engagement: Users are more likely to stay on your site if they can find what they need easily.

Technologies Used

To implement this instant search feature, we will use two main technologies:

• Flask: Flask is a popular web framework for Python. It is simple and lightweight, making it easy to set up and start building web applications quickly. Flask allows us to create routes, handle requests, and serve HTML templates with minimal setup.

• HTMX: This is a powerful JavaScript library that allows us to create dynamic web pages without having to write a lot of JavaScript code. With HTMX, we can update parts of a page based on user actions, like typing in a search box. It makes it easy to load data from the server and display it on the page without a full reload.

Setting Up the Environment

In this section, we will set up the environment for our Flask project, including installing the necessary packages and organizing the project structure.

1. Installing Flask and HTMX

First, you need to install Flask, Flask-SQLAlchemy, and Flask-Migrate. You can do this using pip. Open your terminal and run:

pip install Flask Flask-SQLAlchemy Flask-Migrate

For HTMX, we will include it in our HTML template directly from a CDN.

2. Creating a Virtual Environment

It's a good practice to create a virtual environment for your projects to manage dependencies. Here's how to create one:

python -m venv venv

Next, activate the environment:

# On Windows
venv\Scripts\activate

# On macOS/Linux
source venv/bin/activate

3. Setting Up the Project Structure

Now, set up your project structure as follows:

my_flask_app/
├── core/
│   ├── __init__.py
│   ├── models.py
│   └── routes.py
├── config.py
└── main.py

Let us start with creating the first file - core/__init__.py. This file is the initialization script for the core module of our Flask application. It sets up the Flask app instance and configures it using the settings from the DevelopmentConfig class, and initialize the database and migration system.

from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from config import DevelopmentConfig

# Create the Flask app instance
app = Flask(__name__)

# Load configuration from DevelopmentConfig
app.config.from_object(DevelopmentConfig)

# Initialize SQLAlchemy with the app instance
db = SQLAlchemy(app)

# Initialize Flask-Migrate with the app instance and database
migrate = Migrate(app, db)

# Import routes to register them with the app
from core import routes

Next, we will create the config.py file from where we imported the DevelopmentConfig class. This file contains configuration settings for different environments (development, testing, production). These settings help manage different behaviors and configurations based on where your app is running.

class Config(object):
    DEBUG = False
    TESTING = False
    CSRF_ENABLED = True
    SECRET_KEY = "guess-me"
    SQLALCHEMY_DATABASE_URI = "sqlite:///db.sqlite"
    SQLALCHEMY_TRACK_MODIFICATIONS = False
    BCRYPT_LOG_ROUNDS = 13
    WTF_CSRF_ENABLED = True
    DEBUG_TB_ENABLED = False
    DEBUG_TB_INTERCEPT_REDIRECTS = False

class DevelopmentConfig(Config):
    DEVELOPMENT = True
    DEBUG = True
    WTF_CSRF_ENABLED = False
    DEBUG_TB_ENABLED = True

class TestingConfig(Config):
    TESTING = True
    DEBUG = True
    SQLALCHEMY_DATABASE_URI = "sqlite:///testdb.sqlite"
    BCRYPT_LOG_ROUNDS = 1
    WTF_CSRF_ENABLED = False

class ProductionConfig(Config):
    DEBUG = False
    DEBUG_TB_ENABLED = False

• Config: The base configuration class with default settings.

• DevelopmentConfig: Inherits from Config and overrides development settings.

• TestingConfig: Inherits from Config and overrides settings for testing.

• ProductionConfig: Inherits from Config and overrides production settings.

Finally, we will create the main.py file. This is the entry point of our application. When we run this file, it starts the Flask web server.

from core import app

# Start the Flask app
if __name__ == '__main__':
    app.run(debug=True)

• if __name__ == '__main__': This ensures the Flask app runs only if the script is executed directly (not imported as a module).

• app.run(debug=True): Starts the Flask development server with debug mode enabled, which provides detailed error messages and auto-reloading.

Now that you understand the project files, we can proceed with implementing the instant search functionality. This will involve creating the models and search route, setting up the HTMX-powered front-end, and connecting everything to fetch and display search results dynamically.

Setting up the Database

In this section, we will set up the database for our Flask application. We will use SQLite for simplicity. We will create a model for the data we want to search and seed the database with sample data.

SQLite is a lightweight, disk-based database that doesn’t require a separate server process. It's an excellent choice for development and small projects because it is easy to set up and use.

Creating Model for the Data to Be Searched

We will create a Book model to represent the data in our database. This model will include fields like the book title and author.

Let's create the core/models.py file and add the model there:

from core import db

class Book(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(100), nullable=False)
    author = db.Column(db.String(100), nullable=False)

Applying Migrations Using Flask-Migrate

Before we can seed our database, we need to set up database migrations using Flask-Migrate. This tool helps us manage database changes, such as creating tables and altering schemas, systematically.

Initialize the migrations folder by running the following command in your project directory:

flask db init

This command creates a migrations directory in our project, which will store migration scripts.

Generate a migration script that creates the necessary database tables based on your models:

flask db migrate -m "Initial migration"

This command scans your models and generates a new migration script in the migrations folder.

Apply the migration to create the tables in your database:

flask db upgrade

This command executes the migration script, creating the tables defined by your models in the database. Post this step, you will see an instance/db.sqlite file created.

Seeding Data Into Our Database

Now that we have set up the database and applied the migration, we can proceed with seeding the database. Create a file named seeder.py with the following content:

import csv
from sqlalchemy.exc import IntegrityError

from core import db, app
from core.models import Book


def seed_data():
    with app.app_context():
        # Open the CSV file
        with open("data.csv", newline='', encoding='utf-8') as csvfile:
            reader = csv.DictReader(csvfile)

            # Iterate over the rows in the CSV file
            for row in reader:
                # Create a new Book instance
                book = Book(
                    title=row['Book Name'],
                    author=row['Author Name']
                )

                # Add the book to the session
                db.session.add(book)

            try:
                # Commit the session to write the books to the database
                db.session.commit()
                print("Books added successfully.")
            except IntegrityError as e:
                db.session.rollback()
                print(f"Error occurred: {e}")


if __name__ == "__main__":
    seed_data()

The seeder script is responsible for populating the database with initial data. This is useful for testing and development purposes, allowing you to work with a set of sample data. This script reads data from data.csv, and processes it to insert it into the database.

Note: You can download the data.csv file from here.

To use this script, ensure your data.csv file exists in the same directory as seeder.py. Run the script using Python:

python seeder.py

Setting Up Basic Routing and HTML

In this section, we'll set up a basic route in Flask to serve an index page (index.html) where users can search and display books.

Setting Up Flask Route

Let's set up a Flask route (/) to render an index.html template and display books. For that, create a core/routes.py file and add the following route:

from flask import render_template
from core import app
from core.models import Book

@app.route('/')
def index():
    # Fetch the first 20 books to display by default
    books = Book.query.limit(20).all()
    return render_template("index.html", books=books)

The Flask application handles routing through the @app.route('/') decorator, which directs requests to the root URL (/). When a user visits the homepage, the index() function is invoked. Inside this function, we query the Book model using SQLAlchemy to fetch the first 20 books from the database. These books are then passed as a parameter (books) to the render_template function, which renders the index.html template.

Creating the index.html Template

Create a file named index.html inside a templates directory in your project. The templates directory will lie in the core package. This file will contain the HTML structure for our book search page.

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Book Search</title>
  <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bulma@0.9.4/css/bulma.min.css" />
</head>
<body>
  <section class="section">
    <div class="columns">
      <div class="column is-one-third is-offset-one-third">
        <input type="text" class="input" placeholder="Search" name="query" />
      </div>
    </div>
    <table class="table is-fullwidth">
      <thead>
        <tr>
          <th>ID</th>
          <th>Book Title</th>
          <th>Book Author</th>
        </tr>
      </thead>
      <tbody id="results">
        {% for book in books %}
        <tr>
          <td>{{ book.id }}</td>
          <td>{{ book.title }}</td>
          <td>{{ book.author }}</td>
        </tr>
        {% endfor %}
      </tbody>
    </table>
  </section>
</body>
</html>

This HTML file uses the Bulma CSS framework for styling and includes elements such as an input field for user searches and a table to display book details fetched from the database.

The index.html template utilizes Jinja2 templating to dynamically populate the table rows (<tr>) with book data retrieved from the Flask backend. Each book's ID, title, and author are displayed in the table rows using {{book.id}}, {{ book.title }}, and {{book.author}} respectively.

Running the application

Let's run the application using the following command:

flask run

Once your application is up and running, this is how it will look like:

Adding HTMX for Instant Search

Finally, we will add HTMX to enhance our Flask application with dynamic search capabilities. For this, we'll introduce a new route and modify existing HTML templates.

Creating the Search Route

First, create a new route /search in your Flask application to handle book searches based on user input:

from flask import render_template, request
from core import app
from core.models import Book

@app.route('/search')
def search():
    query = request.args.get("query")
    if query:
        results = Book.query.filter(Book.title.ilike(f"%{query}%") | Book.author.ilike(f"%{query}%")).limit(10).all()
    else:
        results = Book.query.limit(20).all()
    return render_template("search_results.html", results=results)

This route listens for GET requests to /search. It retrieves the search query from the URL parameter using request.args.get("query"). If a query parameter is present, it uses SQLAlchemy's ilike method to perform a case-insensitive search across the title and author columns of the Book table, fetching up to 10 results. If no query parameter is provided, it defaults to fetching the first 20 books from the database. The results are passed to a new search_results.html template for rendering.

Modifying index.html to add HTMX

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Book Search</title>
  <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bulma@0.9.4/css/bulma.min.css" />
  <!-- Include HTMX library -->
  <script src="https://cdn.jsdelivr.net/npm/htmx.org/dist/htmx.min.js"></script>
</head>
<body>
  <section class="section">
    <div class="columns">
      <div class="column is-one-third is-offset-one-third">
        <!-- HTMX-enabled search input -->
        <input
            type="text"
            class="input"
            placeholder="Search"
            name="query"
            hx-get="/search"
            hx-trigger="keyup changed delay:500ms"
            hx-target="#results"
          />
      </div>
    </div>
    <table class="table is-fullwidth">
      <thead>
        <tr>
          <th>ID</th>
          <th>Book Title</th>
          <th>Book Author</th>
        </tr>
      </thead>
      <tbody id="results">
        {% for book in books %}
          <tr>
            <td>{{ book.id }}</td>
            <td>{{ book.title }}</td>
            <td>{{ book.author }}</td>
          </tr>
        {% endfor %}
      </tbody>
    </table>
  </section>
</body>
</html>

The <script> tag imports the HTMX library from a CDN, enabling client-side interactions without requiring complex JavaScript. In addition to that, we enhance the <input> element with HTMX attributes:

• hx-get="/search": Specifies the endpoint (/search) to send GET requests when the user types in the input field.

• hx-trigger="keyup changed delay:500ms": Triggers the search action after a 500ms delay when the user types (keyup) or changes the input (changed).

• hx-target="#results": Updates the content of the element with id="results" with the response from the /search endpoint.

Creating search_results.html Template

Next, we will create a new template search_results.html to display search results:

{% for result in results %}
<tr>
    <td>{{ result.id }}</td>
    <td>{{ result.title }}</td>
    <td>{{ result.author }}</td>
</tr>
{% endfor %}

This template iterates over results, which are passed from the /search route. For each book in results, it generates a table row (<tr>) displaying the book's ID, title, and author.

Demo

Finally, we have implemented instant search with HTMX in our Flask application. Here's what our final application should look like:

Did you notice the delay in the search results? It is called debouncing. It is a programming and web development technique to limit the rate at which a function or event handler is executed. It ensures that a function is only executed after a certain amount of time has passed since the last invocation of the function. In our case, we had set the delay of 500 ms before it hit the /search API. It ensures we do not hit the API for every character the user types.

Conclusion

In this tutorial, we learned how to implement instant search using Flask and HTMX, focusing on enhancing user interaction and performance. By integrating HTMX for AJAX interactions, we enabled dynamic updates to search results without refreshing the entire page. This approach improves user experience by providing real-time feedback and optimizes server load by debouncing search queries.

By mastering these techniques, you're equipped to build responsive web applications that deliver seamless search experiences, combining the flexibility of Flask with the interactivity of HTMX to meet diverse user needs efficiently and effectively.

You can find the code for this tutorial in this repository:

Magic Link Authentication offers a simple yet secure way for users to log in without passwords. This tutorial will walk you through the implementation of Magic Link Authentication using React for the front end, Flask for the back end, and the authentication service provided by Authsignal.

Understanding Magic Link Authentication

Magic link authentication is a convenient and secure authentication method that simplifies users' login process. Instead of entering a username and password, users receive a unique link via email. This link, known as a magic link, grants them access to their account without traditional credentials.

One key difference between magic link authentication and authentication with email verification is the user experience. With magic link authentication, users can authenticate with just a single click. They don't need to remember or enter a password, which can be especially beneficial for users who struggle with password management or find it inconvenient to type in their credentials repeatedly.

While email verification adds an extra layer of security, it may require the user to remember additional credentials or go through multiple steps before accessing their account. If you're interested in learning more about email verification, you can check out my article here to dive deeper into the topic.

How to Configure Authsignal

Authsignal is a service that makes implementing modern authentication methods (like Magic Links and Passkeys) easier. It provides simple tools to integrate secure login methods into your web apps without hassle.

Before proceeding with the tutorial, you need to create an Authsignal account. To do that, you can follow these steps:

First, go to authsignal.com and click on "Create Free Account".

In the next step, create your first tenant. Choose any name for your tenant and select the data storage region.

Next, you need to configure the authenticators you want to use for your application. For example, I have enabled Email Magic Link and Authenticator App (TOTP).

Once you have configured the authenticator, navigate to the API Keys option. Here, you will find your Secret Key, which will be necessary for implementing the authentication.

Application Flow

Let's understand the flow of the application:

Initial Visit

• The user visits the application's user interface. On the user interface, they see a login input box and a signup option. Since the user is new, they opt to sign up.

Signup Flow

• Upon clicking the signup link, the user is directed to a page to enter their chosen username.

• After entering the username and clicking the signup button, the front end triggers a POST API call to /api/signup, sending the username in the request body.

• The backend receives the request and communicates with the Authsignal server for user authentication.

• Authsignal prompts the user to set up Magic Link authentication by entering their email address.

• Authsignal sends a magic link to the provided email address.

• After clicking the magic link, the user is authenticated and redirected to the home page, where they receive a welcome message displaying their email address. The page also includes a logout button.

Login Flow

• The user logs out and returns to the login page.

• Here, the user enters their registered username and clicks the Login button.

• Upon clicking Login, the front end triggers a POST API call to /api/login, passing the username in the request body.

• The backend again communicates with Authsignal for user authentication, prompting the setup of Magic Link authentication.

• The user is directed to a page to enter their email address.

• Authsignal sends a magic link to the provided email address.

• After clicking the magic link, the user is authenticated and redirected to the home page, greeted with a welcome message displaying their email address.

This flow ensures users can sign up using a chosen username, and set up Magic Link authentication via email for both signup and login. Here is a video tutorial to visually aid you in understanding the flow:

How to Set Up Your Backend Server

In this section, I will guide you through how to set up your Flask server for implementing Magic Link Authentication. Before we begin, it's recommended to set up a virtual environment to isolate your project's dependencies. Here's how you can do it:

1. Open your terminal or command prompt.

2. Navigate to your project's directory.

3. Run the following command to create a new virtual environment:

python -m venv myenv

Note: Replace myenv with the desired name for your virtual environment.

4. Activate the virtual environment using the appropriate command for your operating system:

• For Windows:

source myenv/Scripts/activate

• For macOS/Linux:

source myenv/bin/activate

Now that you have your virtual environment set up, let's install the necessary dependencies.

To begin, make sure you have Flask installed, which is a micro web framework for Python. You can install it using the following one-liner:

pip install Flask

Next, we need python-decouple, a library that helps manage configuration settings in separate files. Install it with the following command:

pip install python-decouple

The flask-cors library is a Flask extension that allows for Cross-Origin Resource Sharing (CORS) support in your Flask application.

pip install flask-cors

Finally, we need to install the Python SDK for Authsignal. You can install it with the following command:

pip install authsignal

Now that we have all the necessary dependencies installed, let's create a sample Flask server to get started with Magic Link Authentication. Here's a basic setup to help you get started:

from flask import Flask
from flask_cors import CORS

app = Flask(__name__)
CORS(app, supports_credentials=True)

@app.route("/")
def hello():
    return "Hello, world!"

if __name__ == "__main__":
    app.run(debug=True)

In the next steps, we will integrate AuthSignal and implement the Magic Link Authentication functionality into this server.

How to Set Up Environment Variables

To successfully configure and integrate AuthSignal into your Flask server, you need to set up the following environment variables:

• AUTHSIGNAL_BASE_URL: This variable contains the base URL of the Authsignal server. It allows your server to communicate with Authsignal's authentication service.

• AUTHSIGNAL_SECRET_KEY: This variable contains the secret key associated with your Authsignal project. It is used for secure communication between your server and AuthSignal.

• SECRET_KEY: This variable is a random key used to encrypt the cookies and send them to the browser.

Setting environment variables instead of hardcoding in the code provides improved security by keeping sensitive information, such as API keys and secret keys, separate from the codebase. This reduces the risk of accidental exposure or unauthorized access to these credentials.

To set up these environment variables, you can follow these steps:

1. Open a terminal or command prompt.

2. Navigate to the directory where your Flask server is located.

3. Create a .env file and export the environment variables using the following commands:

export AUTHSIGNAL_BASE_URL=<base_url>
export AUTHSIGNAL_SECRET_KEY=<secret_key>
export SECRET_KEY=<random-secret-key>

Make sure to replace <base_url>, <secret_key>, and <random-secret-key> with the appropriate values for your Authsignal project. You can find the values for these environment variables in the API Keys section of the Authsignal dashboard as explained earlier.

Note: The method for setting environment variables can vary depending on your operating system. The above commands are applicable for Unix-based systems. For Windows, you can use the set command instead of export.

To export the variables added in the .env file, you can use the following command in the terminal:

source .env

By properly setting these environment variables, your Flask server can securely communicate with Authsignal and implement the Magic Link Authentication functionality.

How to Initialize the Authsignal Client

To integrate Authsignal into your Flask server and implement Magic Link Authentication, you need to initialize the Authsignal client. Here's an example of how you can do this:

from flask import Flask
from flask_cors import CORS
import authsignal.client
from decouple import config

app = Flask(__name__)
CORS(app)

AUTHSIGNAL_BASE_URL = config("AUTHSIGNAL_BASE_URL")
AUTHSIGNAL_SECRET_KEY = config("AUTHSIGNAL_SECRET_KEY")
SECRET_KEY = config("SECRET_KEY")

authsignal_client = authsignal.Client(
    api_key=AUTHSIGNAL_SECRET_KEY,
    api_url=AUTHSIGNAL_BASE_URL
)

In this code snippet, we first import the authsignal.client, the Python SDK for Authsignal. We also import config from python-decouple to retrieve the environment variables.

We retrieve the environment variables AUTHSIGNAL_BASE_URL, AUTHSIGNAL_SECRET_KEY and SECRET_KEY using config from python-decouple.

Finally, we initialize the authsignal.Client by passing in the API key AUTHSIGNAL_SECRET_KEY and the base URL of the Authsignal server AUTHSIGNAL_BASE_URL.

By initializing the Authsignal client, we are ready to implement the Magic Link Authentication functionality in our Flask server.

Authsignal Actions

Authsignal allows you to create actions to track and manage user interactions in your application. Actions are events that can be triggered by users, such as signing up or logging in. By creating custom actions, you can have more control over the authentication process and implement specific authentication methods like Magic Link Authentication.

To create an action on your Authsignal dashboard, follow these steps:

1. Click on "Actions" in your Authsignal dashboard.

2. Click on "Configure a new action" to create a new action.

3. Enter a name for the action that describes its purpose or the user interaction it represents.

4. Next, you can configure the rule for the action. In our case, since we want to implement Magic Link Authentication, we will add a rule to challenge users with Email Magic Link. This will send a magic link to the user's email for authentication.

5. Save the action to apply the rule and make it active.

Here is a video demonstrating the process of creating an Authsignal action and configuring it for Magic Link Authentication:

You can create two actions – "signUp" and "signIn". In the next steps, we will make use of these actions.

How to Create the Required Routes

Finally, to implement the Magic Link Authentication, we need to create three routes: /api/signup, /api/login, /api/callback, and /api/user.

/api/signup Route

The /api/signup route is responsible for allowing the users to register in our application. Here's how we implement it:

@app.route('/api/signup', methods=['POST'])
def signup():
    username = request.json.get('username')
    if not username:
        return jsonify({'error': 'Missing username parameter'}), 400

    response = authsignal_client.track(
        user_id=username,
        action="signUp",
        payload={
            "user_id": username,
            "redirectUrl": "http://localhost:5000/api/callback"
        }
    )
    return jsonify(response), 200

In this implementation, the route expects a JSON payload containing the username parameter. It then uses the authsignal_client to track the user's signUp action and generate a Magic Link. The track method lets you record actions performed by users and initiate challenges. The redirectUrl specifies the URL where the user will be redirected after they have been authenticated. We will create this API next.

/api/callback Route

The /api/callback route handles the callback URL where the user is redirected after verifying themselves. Here's the implementation for this route:

@app.route('/api/callback', methods=['GET'])
def callback():
    token = request.args.get('token')
    challenge_response = authsignal_client.validate_challenge(token)

    if challenge_response["state"] == 'CHALLENGE_SUCCEEDED':
        encoded_token = jwt.encode(
            payload={"username": challenge_response["user_id"]},
            key=SECRET_KEY,
            algorithm="HS256"
        )
        response = redirect('http://localhost:3000/')
        response.set_cookie(
            key='auth-session',
            value=encoded_token,
            secure=False,
            path='/'
        )
        return response

    return redirect("/")

When the users are redirected, Authsignal adds the JWT token in the URL as a token query parameter.

In this implementation, the route retrieves the token parameter from the query string. It then uses the authsignal_client to validate the challenge and check if the authentication was successful.

If the authentication succeeds, we encode a JSON Web Token (JWT). The token payload includes the username obtained from the challenge response. It uses the SECRET_KEY and the HS256 algorithm for encryption.

Next, the user is redirected to the home page (http://localhost:3000/), and a auth-session cookie is set with the encoded token for further user identification.

Note that the token returned from Authsignal in the redirect is not intended to be used as a session token. It just contains information about the challenge so that we can determine if the challenge was successful.

/api/login Route

The /api/login route is responsible for allowing the users to log into the application. Here’s the implementation for the route:

@app.route('/api/login', methods=['POST'])
def login():
    username = request.json.get('username')
    if not username:
        return jsonify({'error': 'Missing username parameter'}), 400

    response = authsignal_client.track(
        user_id=username,
        action="signIn",
        payload={
            "user_id": username,
            "redirectUrl": "http://localhost:5000/api/callback"
        }
    )
    return jsonify(response), 200

The route is configured to handle POST requests on the /api/login endpoint. Upon receiving a POST request, the route first extracts the provided username from the JSON payload sent with the request. It ensures that the username is present. If not, it promptly returns a 400 error response indicating a missing username parameter.

Similar to the signup flow, it then uses the authsignal_client to track the user's signIn action and generate a Magic Link. The redirectUrl specifies the URL where the user will be redirected after they have been authenticated.

/api/user Route

The /api/user route is responsible for retrieving user information. Here's the implementation for this route:

@app.route("/api/user", methods=['GET'])
def user():
    token = request.cookies.get('auth-session')
    decoded_token = jwt.decode(token, SECRET_KEY, algorithms=["HS256"])
    username = decoded_token.get('username')
    response = authsignal_client.get_user(user_id=username)
    return jsonify({"username": username, "email": response["email"]}), 200

In this implementation, the GET endpoint starts by extracting the auth-session cookie from the incoming request. Then it decodes the JWT using the jwt.decode method, utilizing the SECRET_KEY as the secret key for decoding.

The decoded token provides the username of the user. It then uses the authsignal_client to retrieve user information based on the provided userId. It then returns a JSON response with the username and email information.

By implementing these routes, we will be able to handle the basic authentication process and retrieve user information in our Flask server.

How to Set Up a New Frontend React Project

Let's set up our front-end project in this section. This will also include setting up routing in the application.

Start by initializing a new React project using create-react-app or any preferred method (like Vite, for example, which is a more modern way to set up a React app). This command sets up the basic structure for your React application.

npx create-react-app magic-link-auth
cd magic-link-auth

Once the project is created and you're inside the project directory, install the required dependencies. Here, we need react-router-dom for handling routing and bootstrap for easy styling.

npm install react-router-dom bootstrap

Import Bootstrap CSS

Bootstrap provides pre-styled components and utilities for easier and faster styling of your application.

In the index.js file, import Bootstrap:

import React from 'react';
import ReactDOM from 'react-dom/client';
import App from './App';
import reportWebVitals from './reportWebVitals';
import "bootstrap/dist/css/bootstrap.min.css"; // Import Bootstrap CSS

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

// If you want to start measuring performance in your app, pass a function
// to log results (for example: reportWebVitals(console.log))
// or send to an analytics endpoint. Learn more: https://bit.ly/CRA-vitals
reportWebVitals();

Additionally, we will write some custom CSS. Replace the code in the index.css file with the following code:

html,
body {
  padding: 0;
  margin: 0;
  font-family: -apple-system, BlinkMacSystemFont, Segoe UI, Roboto, Oxygen,
    Ubuntu, Cantarell, Fira Sans, Droid Sans, Helvetica Neue, sans-serif;
}

* {
  box-sizing: border-box;
}

main {
  padding: 5rem 0;
  flex: 1;
  display: flex;
  flex-direction: column;
  justify-content: center;
  align-items: center;
}

code {
  background: #fafafa;
  border-radius: 5px;
  padding: 0.75rem;
  font-family: Menlo, Monaco, Lucida Console, Courier New, monospace;
}

input[type="button"] {
  border: none;
  background: cornflowerblue;
  color: white;
  padding: 12px 24px;
  margin: 8px;
  font-size: 18px;
  border-radius: 8px;
  cursor: pointer;
}

Similarly, replace the code in the App.css with the following code:

.mainContainer {
  flex-direction: column;
  display: flex;
  align-items: center;
  justify-content: center;
  height: 100vh;
}

.titleContainer {
  display: flex;
  flex-direction: column;
  font-size: 48px;
  font-weight: bolder;
  align-items: center;
  justify-content: center;
}

.resultContainer,
.historyItem {
  flex-direction: row;
  display: flex;
  width: 400px;
  align-items: center;
  justify-content: space-between;
}

.historyContainer {
  flex-direction: column;
  display: flex;
  height: 200px;
  align-items: center;
  flex-grow: 5;
  justify-content: flex-start;
}

.buttonContainer {
  display: flex;
  flex-direction: column;
  align-items: center;
  justify-content: center;
  height: 260px;
}

.inputContainer {
  display: flex;
  flex-direction: column;
  align-items: flex-start;
  justify-content: center;
}

.inputContainer>.errorLabel {
  color: red;
  font-size: 16px;
  text-align: center;
}

.inputBox {
  height: 48px;
  width: 400px;
  font-size: medium;
  border-radius: 8px;
  border: 1px solid grey;
  padding-left: 8px;
}

.inputButton {
  height: 48px;
  width: 400px;
}

Set Up Routing

Routing in React applications helps navigate between different views or pages. react-router-dom simplifies this process.

In your App.js file, configure the routing:

import React from "react";
import { BrowserRouter, Routes, Route } from "react-router-dom";
import Dashboard from "./pages/Dashboard";
import Register from "./pages/Register";
import Login from "./pages/Login";
import "./App.css";
import "./index.css";

const App = () => {
  return (
    <BrowserRouter>
      <Routes>
        <Route path="/" element={<Dashboard />} />
        <Route path="/login" element={<Login />} />
        <Route path="/signup" element={<Register />} />
      </Routes>
    </BrowserRouter>
  );
};

export default App;

It defines an App component that encapsulates the entire application structure within a <BrowserRouter> component. Inside <Routes>, we define three <Route> components: one for the root path / rendering the Dashboard component, and two for the /login and /signup paths, rendering the Login and Register components respectively.

This setup enables navigation between different views based on URL paths, allowing users to access specific components when they visit corresponding routes within the application.

In the upcoming sections, we will set up the above-mentioned three components.

How to Set Up the Components

In the previous section, we imported two components from the src/pages folder. Let's create a pages folder inside the src folder, and then we can start creating the components.

Register Component

Let’s create a Register.jsx file inside the pages folder. The Register component allows users to register within our application.

import React, { useState, useEffect } from "react";
import { useNavigate, Link } from "react-router-dom";

const Register = () => {
  const [username, setUsername] = useState("");
  const [usernameError, setUsernameError] = useState("");

  const navigate = useNavigate();

  useEffect(() => {
    const isAuthenticated = checkCookies();
    if (isAuthenticated) {
      navigate("/");
    }
  }, [navigate]);

  const checkCookies = () => {
    const authSessionCookie = document.cookie.match("auth-session=([^;]+)");

    return !!authSessionCookie;
  };

  const onButtonClick = () => {
    setUsernameError("");

    if ("" === username) {
      setUsernameError("Username is mandatory!");
      return;
    }

    signup();
  };

  const signup = async () => {
    const response = await fetch("http://localhost:5000/api/login", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        username,
      }),
      credentials: "include",
    });

    const { url } = await response.json();

    // Redirect to verification URL
    window.location.href = url;
  };

  return (
    <div className={"mainContainer"}>
      <div className={"titleContainer"}>
        <div>Sign Up</div>
      </div>
      <br />
      <div className={"inputContainer"}>
        <input
          value={username}
          placeholder="Enter your username"
          onChange={(e) => setUsername(e.target.value)}
          className={"inputBox"}
        />
        <label className="errorLabel text-center">{usernameError}</label>
      </div>
      <br />
      <div className={"inputContainer"}>
        <input
          className={"inputButton"}
          type="button"
          onClick={onButtonClick}
          value={"Sign Up"}
        />
      </div>
      <div>
        Existing User? <Link to="/login">Login here</Link>
      </div>
    </div>
  );
};

export default Register;

The component initializes state variables using useState to manage the visibility of the usernameError and store the username input in userName. It also initializes the navigate function from useNavigate to handle navigation within the application.

We use the useEffect hook to check for authentication cookies when the component mounts. It calls the checkCookies function, which checks for the existence of cookies that we had set from the backend server. If auth-session cookie is found, the user is automatically redirected to the root URL using navigate("/").

Clicking the “Sign Up” button triggers the onButtonClick function. It first checks whether the user has entered the username. If not, it shows an error message using the usernameError. If the user has entered the username, it calls the signup function.

The signup performs an asynchronous POST request to the /api/signup endpoint with the provided username. Upon successful response, it redirects the user to the received verification URL by changing window.location.href.

The JSX returned by the component defines the UI layout that looks like the below:

It includes an input field for the users to enter their username and a "Sign Up" button. Below the button, we have a link to the Login page for the existing users to log in.

Login Component

We have kept the login page similar to the signup page for simplicity. Hence, the Login component is pretty much the same as the Register component.

import React, { useState, useEffect } from "react";
import { useNavigate, Link } from "react-router-dom";

const Login = () => {
  const [username, setUsername] = useState("");
  const [usernameError, setUsernameError] = useState("");

  const navigate = useNavigate();

  useEffect(() => {
    const isAuthenticated = checkCookies();
    if (isAuthenticated) {
      navigate("/");
    }
  }, [navigate]);

  const checkCookies = () => {
    const authSessionCookie = document.cookie.match("auth-session=([^;]+)");

    return !!authSessionCookie;
  };

  const onButtonClick = () => {
    setUsernameError("");

    if ("" === username) {
      setUsernameError("Username is mandatory!");
      return;
    }

    login();
  };

  const login = async () => {
    const response = await fetch("http://localhost:5000/api/login", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        username,
      }),
      credentials: "include",
    });

    const { url } = await response.json();

    // Redirect to verification URL
    window.location.href = url;
  };

  return (
    <div className={"mainContainer"}>
      <div className={"titleContainer"}>
        <div>Login</div>
      </div>
      <br />
      <div className={"inputContainer"}>
        <input
          value={username}
          placeholder="Enter your username"
          onChange={(ev) => setUsername(ev.target.value)}
          className={"inputBox"}
        />
        <label className="errorLabel text-center">{usernameError}</label>
      </div>
      <br />
      <div className={"inputContainer"}>
        <input
          className={"inputButton"}
          type="button"
          onClick={onButtonClick}
          value={"Log in"}
        />
      </div>
      <div>
        New User? <Link to="/signup">Sign up here</Link>
      </div>
    </div>
  );
};

export default Login;

The only significant difference other than the text and button in the UI here is that we will be making the API call to the /api/login route when the users hit the Login button.

The UI looks like below:

Dashboard Component

The Dashboard component in our application serves as the interface for authenticated users, displaying a welcome message with the user’s email and enabling user logout functionality.

Let’s create a Dashboard.jsx file inside the pages folder.

import React, { useState, useEffect } from "react";
import { useNavigate } from "react-router-dom";

const Dashboard = () => {
  const [userEmail, setUserEmail] = useState("");
  const navigate = useNavigate();

  useEffect(() => {
    const checkCookies = async () => {
      const authSessionCookie = document.cookie.match("auth-session=([^;]+)");

      if (!authSessionCookie) {
        navigate("/auth");
        return false;
      }

      return true;
    };

    const fetchData = async () => {
      const cookiesValid = await checkCookies();
      if (!cookiesValid) return;

      try {
        const response = await fetch("http://localhost:5000/api/user", {
          method: "GET",
          headers: {
            "Content-Type": "application/json",
          },
          credentials: "include"
        });

        if (!response.ok) {
          throw new Error("Failed to fetch user data");
        }

        const data = await response.json();
        setUserEmail(data.email);
      } catch (error) {
        navigate("/auth");
      }
    };

    fetchData();
  }, [navigate]);

  const handleLogout = () => {
    document.cookie = `auth-session=; max-age=0`;
    navigate("/auth");
  };

  return (
    <div className="d-flex justify-content-center align-items-center vh-100">
      <main className="px-3 text-center">
        <h1>Welcome Home!</h1>
        <p className="lead">You're logged in as {userEmail}!</p>
        <div className="d-flex justify-content-center">
          <button
            className="btn btn-lg btn-dark fw-bold border-white bg-dark"
            onClick={handleLogout}
          >
            Log Out
          </button>
        </div>
      </main>
    </div>
  );
};

export default Dashboard;

Upon component mounting or when navigate changes (a dependency of useEffect), the effect runs. It begins by defining two asynchronous functions. The first, checkCookies, verifies the presence of auth-session cookies. If it is missing, it redirects the user to the authentication route.

The second function, fetchData, is responsible for fetching user data. It checks the validity of cookies using checkCookies. Upon verification, it sends a GET request to our back-end API endpoint. Upon successful response, it updates the userEmail state with the user's email fetched from the API data.

If any error occurs during this process, such as failing to fetch user data, it redirects the user back to the authentication route.

The JSX returned by the component renders a simple dashboard layout.

The displayed content includes a welcoming message and the currently logged-in user's email. There's also a "Log Out" button which, upon clicking, initiates the logout process by triggering the handleLogout function. It removes the auth-session cookies by setting their max-age to 0, effectively expiring them. Afterward, it redirects the user to the authentication route.

How to Run the Application

You can find the code of the final application in this GitHub repository.

To run your backend application, run python app.py from your back-end folder in your terminal. This will start your back-end server on port 5000. Next, run the frontend application using the npm start command. This will start your frontend application on the port 3000.

Wrapping Up

In this tutorial, you learned how to use Authsignal to implement basic user authentication with email verification through magic links.

Authsignal makes handling users and keeping things safe easier, letting developers focus on improving apps. It also removes the overhead of remembering another password for the users of the application.

To learn more about Authsignal, visit the Authsignal documentation.

Should you have any issues or questions related to the tutorial, then feel free to reach out to me on Twitter.

Think of it as a shield for your accounts. Passwords can sometimes be guessed or stolen, but with 2FA, even if someone gets your password, they'd still need that extra code or device to get in. It's an extra step that makes your accounts much harder for hackers to break into.

So, let's explore how to set up this extra layer of protection using PyOTP and Google Authenticator in your Flask app.

Overview of PyOTP and Google Authenticator

PyOTP is a Python library that's incredibly handy for generating Time-based One-Time Passwords (TOTP) and HMAC-based One-Time Passwords (HOTP). Its primary role revolves around creating these unique, time-sensitive codes that add an extra layer of security to user accounts.

By integrating PyOTP into your Flask application, you can easily implement Two-Factor Authentication (2FA) by generating and verifying these OTPs.

If you're new to PyOTP or would like a refresher on its functionalities, I recommend reviewing my previous guide on PyOTP. This understanding will be beneficial as we get into the integration of PyOTP within your Flask application for Two-Factor Authentication (2FA).

Google Authenticator, on the other hand, stands out as one of the most widely used OTP generator apps available. It functions as a secure platform for generating time-based OTPs, compatible with various services and applications supporting 2FA. Users can easily set up Google Authenticator on their devices to generate these time-sensitive codes, adding an extra level of security to their accounts.

Two-Factor Authentication Workflow in Our Application

Here's a breakdown of the flow of two-factor authentication in our application:

1. Registration with 2FA Setup: When users sign up on our website, they're prompted to set up an extra layer of security—2FA. This involves scanning a QR code using an authenticator app, such as Google Authenticator, to link their account securely.

2. Login Initiation: When users return to log in, they start by entering their usual email/username and password combo to access their account.

3. Extra Security Check: Before granting access, our website throws in an additional hurdle: users need to provide an OTP (One-Time Password) displayed on their authenticator app. This ensures they're not just entering the password but also confirming their identity with a unique, time-sensitive code.

4. Validation and Authorization: The user inputs the received OTP into our platform. The system then double-checks this OTP against the expected code, validating the information. If the OTP matches, it's like handing over the secret handshake, granting the user access to their account.

This seamless back-and-forth between passwords, authenticator apps, and unique codes ensures that only the rightful account owner can access the precious content behind the digital doors of your website.

If you also enjoy visual learning, here's a fancy video showing how the app does its thing.

Now, let's get to some coding!

Prerequisites

Before you get started with the tutorial, make sure you have the following requirements satisfied:

• Working knowledge of Python

• Python 3.8+ installed on your system

• Basic knowledge of Flask and Flask Blueprints

• Knowledge of basic authentication in Flask (optional)

Get Your Tools Ready

You'll need a few external libraries for this project. Let's learn more about them and install them one by one.

But before we install them, let's create a virtual environment and activate it.

First, start with creating the project directory and navigating to it like this:

mkdir flask-two-factor-auth
cd flask-two-factor-auth

We are going to create a virtual environment using venv. Python now ships with a pre-installed venv library. So, to create a virtual environment, you can use the below command:

python -m venv env

The above command will create a virtual environment named env. Now, we need to activate the environment using this command:

source env/Scripts/activate

To verify if the environment has been activated or not, you can see (env) in your terminal. Now, we can install the libraries.

• Flask is a simple, easy-to-use microframework for Python that helps you build scalable and secure web applications.

• Flask-Login provides user session management for Flask. It handles the common tasks of logging in, logging out, and remembering your users’ sessions over extended periods.

• Flask-Bcrypt is a Flask extension that provides bcrypt hashing utilities for your application.

• Flask-WTF is a simple integration of Flask and WTForms that helps you create forms in Flask.

• Flask-Migrate is an extension that handles SQLAlchemy database migrations for Flask applications using Alembic. The database operations are made available through the Flask command-line interface.

• Flask-SQLAlchemy is an extension for Flask that adds support for SQLAlchemy to your application. It helps you simplify things using SQLAlchemy with Flask by giving you useful defaults and extra helpers that make it easier to perform common tasks.

• PyOTP helps you generate OTPs using Time-based OTP (TOTP) and HMAC-based OTP (HOTP) algorithms effortlessly.

• QRCode helps you generate QR Codes in Python

• Python Decouple helps you use environment variables in your Python project.

To install the above-mentioned libraries all in one go, run the following command:

pip install Flask Flask-Login Flask-Bcrypt Flask-WTF FLask-Migrate Flask-SQLAlchemy pyotp qrcode python-decouple

How to Set Up the Project

Let’s start by creating a src directory:

mkdir src

The first file will be the __init__.py file for the project:

from decouple import config
from flask import Flask
from flask_bcrypt import Bcrypt
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config.from_object(config("APP_SETTINGS"))

bcrypt = Bcrypt(app)
db = SQLAlchemy(app)
migrate = Migrate(app, db)

# Registering blueprints
from src.accounts.views import accounts_bp
from src.core.views import core_bp

app.register_blueprint(accounts_bp)
app.register_blueprint(core_bp)

In the above script, we created a Flask app called app . We use the __name__ argument to indicate the app's module or package so that Flask knows where to find other files such as templates. We also set the configuration of the app using an environment variable called APP_SETTINGS. We'll export it later.

To use Flask-Bcrypt, Flask-SQLAlchemy, and Flask-Migrate in our application, we just need to create objects of the Bcrypt, SQLAlchemy and Migrate classes from the flask_bcrypt, flask_sqlalchemy and, flask_migrate libraries, respectively.

We've also registered blueprints called accounts_bp and core_bp in the application. We'll define them later in the tutorial.

In the root directory of the project (that is, outside the src directory), create a file called config.py. We'll store the configurations for the project in this file. Within the file, add the following content:

from decouple import config

DATABASE_URI = config("DATABASE_URL")
if DATABASE_URI.startswith("postgres://"):
    DATABASE_URI = DATABASE_URI.replace("postgres://", "postgresql://", 1)


class Config(object):
    DEBUG = False
    TESTING = False
    CSRF_ENABLED = True
    SECRET_KEY = config("SECRET_KEY", default="guess-me")
    SQLALCHEMY_DATABASE_URI = DATABASE_URI
    SQLALCHEMY_TRACK_MODIFICATIONS = False
    BCRYPT_LOG_ROUNDS = 13
    WTF_CSRF_ENABLED = True
    DEBUG_TB_ENABLED = False
    DEBUG_TB_INTERCEPT_REDIRECTS = False
    APP_NAME = config("APP_NAME")


class DevelopmentConfig(Config):
    DEVELOPMENT = True
    DEBUG = True
    WTF_CSRF_ENABLED = False
    DEBUG_TB_ENABLED = True


class TestingConfig(Config):
    TESTING = True
    DEBUG = True
    SQLALCHEMY_DATABASE_URI = "sqlite:///testdb.sqlite"
    BCRYPT_LOG_ROUNDS = 1
    WTF_CSRF_ENABLED = False


class ProductionConfig(Config):
    DEBUG = False
    DEBUG_TB_ENABLED = False

In the above script, we have created a Config class and defined various attributes inside that. Also, we have created different child classes (as per different stages of development) that inherit the Config class.

Notice that we're using a few environment variables like SECRET_KEY, DATABASE_URL, and APP_NAME. Create a file named .env in the root directory and add the following content there:

export SECRET_KEY=fdkjshfhjsdfdskfdsfdcbsjdkfdsdf
export DEBUG=True
export APP_SETTINGS=config.DevelopmentConfig
export DATABASE_URL=sqlite:///db.sqlite
export FLASK_APP=src
export FLASK_DEBUG=1
export APP_NAME="Flask User Authentication App"

Apart from the SECRET_KEY , DATABASE_URL and APP_NAME, we've also exported APP_SETTINGS, DEBUG, FLASK_APP, and FLASK_DEBUG.

The APP_SETTINGS refers to one of the classes we created in the config.py file. We set it to the current stage of the project.

The value of FLASK_APP is the name of the package we have created. Since the app is in the development stage, you can set the values of DEBUG and FLASK_DEBUG to True and 1, respectively.

Run the following command to export all the environment variables from the .env file:

source .env

Next, we'll create a CLI application of the app so that we can later add custom commands if required.

Create a manage.py file in the root directory of the application and add the following code:

from flask.cli import FlaskGroup

from src import app

cli = FlaskGroup(app)


if __name__ == "__main__":
    cli()

Now, your basic application is ready. You can run it using the following command:

python manage.py run

Your file structure should look like below as of now:

flask-two-factor-auth/
├── src/
│   └── __init__.py
├── .env
├── config.py
└── manage.py

How to Create Blueprints for Accounts and Core

As mentioned earlier, you'll use the concepts of blueprints in the project. Let's create two blueprints – accounts_bp and core_bp – in this section.

First, create a directory called accounts like this:

mkdir accounts
cd accounts

Next, add an empty __init__.py file to convert it into a Python package. Now, create a views.py file inside the package where you'll store all your routes related to user authentication.

touch __init__.py views.py

Add the following code to the views.py file:

from flask import Blueprint

accounts_bp = Blueprint("accounts", __name__)

In the above script, you have created a blueprint called accounts_bp for the accounts package.

Similarly, you can create a core package in the root directory, and add a views.py file.

mkdir core
cd core
touch __init__.py views.py

Now, add the following code to the views.py file:

from flask import Blueprint

core_bp = Blueprint("core", __name__)

Note: If you're new to Flask Blueprints, make sure you go through this tutorial to learn more about how it works.

Now, your file structure should look like what you see below:

flask-two-factor-auth/
├── src/
│   ├── accounts/
│   │   ├── __init__.py
│   │   └── views.py
│   ├── core/
│   │   ├── __init__.py
│   │   └── views.py
│   └── __init__.py
├── .env
├── config.py
└── manage.py

How to Create a User Model

Let's create a models.py file inside the accounts package.

touch src/accounts/models.py

Inside the models.py file, add the following code:

from datetime import datetime

import pyotp
from flask_login import UserMixin

from src import bcrypt, db
from config import Config


class User(db.Model):

    __tablename__ = "users"

    id = db.Column(db.Integer, primary_key=True)
    username = db.Column(db.String, unique=True, nullable=False)
    password = db.Column(db.String, nullable=False)
    created_at = db.Column(db.DateTime, nullable=False)
    is_two_factor_authentication_enabled = db.Column(
        db.Boolean, nullable=False, default=False)
    secret_token = db.Column(db.String, unique=True)

    def __init__(self, username, password):
        self.username = username
        self.password = bcrypt.generate_password_hash(password)
        self.created_at = datetime.now()
        self.secret_token = pyotp.random_base32()

    def get_authentication_setup_uri(self):
        return pyotp.totp.TOTP(self.secret_token).provisioning_uri(
            name=self.username, issuer_name=Config.APP_NAME)

    def is_otp_valid(self, user_otp):
        totp = pyotp.parse_uri(self.get_authentication_setup_uri())
        return totp.verify(user_otp)

    def __repr__(self):
        return f"<user {self.username}>"

In the above code, you created a User model by inheriting the db.Model class. The User model consists of the following fields:

• id: stores the primary key for the users table

• username: stores the username of the user

• password: stores the hashed password of the user

• created_at: stores the timestamp when the user was created

• is_two_factor_authentication_enabled: boolean flag that stores whether the user has activated two-factor authentication. Default value is False.

• secret_token: stores a unique token generated for each user, essential for implementing two-factor authentication.

The constructor initializes the User object upon instantiation by accepting username and password parameters. It hashes the provided password using bcrypt.generate_password_hash(password), records the current timestamp as the created_at value, and generates a unique secret_token using pyotp.random_base32() for 2FA setup.

The get_authentication_setup_uri() method generates a setup URI used by authenticator apps like Google Authenticator. It constructs a URI containing the user's username and the application's name (Config.APP_NAME) necessary for setting up two-factor authentication. The basic format of the URI is:

otpauth://totp/Example:alice@google.com?secret=JBSWY3DPEHPK3PXP&issuer=Example

where alice@google.com is the username of the user and Example is the application's name.

Next up, the is_otp_valid() method verifies the one-time password (OTP) entered by the user during login. It parses the setup URI generated earlier, checks the validity of the provided OTP (user_otp), and returns True if the OTP matches, ensuring secure authentication.

Finally, the __repr__ method provides a string representation of the User object, displaying the associated username when an instance of the class is printed or represented as a string.

How to Add Flask-Login

The most important part of Flask-Login is the LoginManager class that lets your application and Flask-Login work together.

In the src/__init__.py file, add the following code:

from decouple import config
from flask import Flask
from flask_login import LoginManager # Add this line
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
app.config.from_object(config("APP_SETTINGS"))

login_manager = LoginManager() # Add this line
login_manager.init_app(app) # Add this line
db = SQLAlchemy(app)
migrate = Migrate(app, db)

# Registering blueprints
from src.accounts.views import accounts_bp
from src.core.views import core_bp

app.register_blueprint(accounts_bp)
app.register_blueprint(core_bp)

In the above script, we created and initialized the login manager in our app.

Next, we need to provide a user_loader callback. This callback is used to reload the user object from the user ID stored in the session. It should take the ID of a user, and return the corresponding user object.

from src.accounts.models import User

@login_manager.user_loader
def load_user(user_id):
    return User.query.filter(User.id == int(user_id)).first()

The User model should implement the following properties and methods:

• is_authenticated: This property returns True if the user is authenticated.

• is_active: This property returns True if this is an active user (the account is activated)

• is_anonymous: This property returns True if this is an anonymous user (actual users return False).

• get_id(): This method returns a string that uniquely identifies this user, and can be used to load the user from the user_loader callback.

Now, we don't need to implement these explicitly. Instead, the Flask-Login provides a UserMixin class that contains the default implementations for all of these properties and methods. We just need to inherit it in the following way:

from datetime import datetime

from flask_login import UserMixin # Add this line

from src import bcrypt, db


class User(UserMixin, db.Model): # Change this line
    ....

We can also customize the default login process in the src/__init__.py file.

The name of the login view can be set as LoginManager.login_view. The value refers to the function name that will handle the login process.

login_manager.login_view = "accounts.login"

To customize the message category, set LoginManager.login_message_category:

login_manager.login_message_category = "danger"

How to Add Templates and Static Files

Let's create a CSS file called styles.css inside the src/static folder:

.error {
  color: red;
  margin-bottom: 5px;
  text-align: center;
}

a {
  text-decoration: none;
}

Let's also create the basic templates inside the src/templates folder. Create a _base.html file and add the following code:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <title>Two Factor Authentication</title>
    <!-- meta -->
    <meta name="description" content="">
    <meta name="author" content="">
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <!-- styles -->
    <!-- CSS only -->
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-T3c6CoIi6uLrA9TneNEoa7RxnatzjcDSCmG1MXxSR1GAsXEV/Dwwykc2MPK8M2HN" crossorigin="anonymous">
    <link rel="stylesheet" href="{{url_for('static', filename="styles.css")}}">
    {% block css %}{% endblock %}
  </head>
  <body>

    {% include "navigation.html" %}

    <div class="container">

      <br>

      <!-- messages -->
      {% with messages = get_flashed_messages(with_categories=true) %}
      {% if messages %}
      <div class="row">
        <div class="col-md-4"></div>
        <div class="col-md-4">
          {% for category, message in messages %}
          <div class="alert alert-{{ category }} alert-dismissible fade show" role="alert">
           {{message}}
           <button type="button" class="btn-close" data-bs-dismiss="alert" aria-label="Close"></button>
          </div>
          {% endfor %}
        </div>
        <div class="col-md-4"></div>
      </div>
      {% endif %}
      {% endwith %}

      <!-- child template -->
      {% block content %}{% endblock %}

    </div>

    <!-- scripts -->
    <script src="https://code.jquery.com/jquery-3.7.1.min.js" integrity="sha256-/JqT3SQfawRcv/BIHPThkBvs0OEvtFFmqPF/lYI/Cxo=" crossorigin="anonymous"></script>
    <!-- JavaScript Bundle with Popper -->
    <script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.11.8/dist/umd/popper.min.js" integrity="sha384-I7E8VVD/ismYTF4hNIPjVp/Zjvgyol6VFvRkX/vR+Vc4jQkC+hVqc2pM8ODewa9r" crossorigin="anonymous"></script>
    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.min.js" integrity="sha384-BBtl+eGJRgqQAUMxJ7pMwbEyER4l1g+O15P+16Ep7Q9Q+zqX6gSbd85u4mG4QzX+" crossorigin="anonymous"></script>
    {% block js %}{% endblock %}
  </body>
</html>

The _base.html is the parent HTML file that will be inherited by the other templates. We have added Bootstrap 5 support in the above file. We are also making use of Flask Flashes to show Bootstrap alerts in the app.

Let's also create a navigation.html file that contains the navbar of the app:

<!-- Navigation -->
<nav class="navbar bg-dark navbar-expand-lg bg-body-tertiary p-3" data-bs-theme="dark">
  <div class="container-fluid">
    <a class="navbar-brand" href="{{ url_for('core.home') }}">Two-Factor Authentication App</a>
    <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation">
      <span class="navbar-toggler-icon"></span>
    </button>
    <div class="collapse navbar-collapse" id="navbarSupportedContent">
      {% if current_user.is_authenticated %}
      <a href="{{ url_for('accounts.logout') }}"><button type="button" class="btn btn-danger me-2">Logout</button></a>
      {% endif %}
    </div>
  </div>
</nav>

Note that we have not yet created the views used above.

How to Create the Homepage

In this section, we'll first create a view function for the homepage inside the core/views.py file. Add the following code there:

from flask import Blueprint, render_template
from flask_login import login_required

core_bp = Blueprint("core", __name__)


@core_bp.route("/")
@login_required
def home():
    return render_template("core/index.html")

Notice that we have used the blueprint to add the route. We also added a @login_required middleware to prevent access for unauthenticated users.

Next, let's create an index.html file inside the templates/core folder, and add the following code:

{% extends "_base.html" %}
{% block content %}

<h1 class="text-center">Welcome {{current_user.username}}!</h1>

{% endblock %}

The HTML page will just have a welcome message for authenticated users.

Your file structure as of now should look like below:

flask-two-factor-auth/
├── src/
│   ├── accounts/
│   │   ├── __init__.py
│   │   └── views.py
│   ├── core/
│   │   ├── __init__.py
│   │   └── views.py
│   ├── static/
│   │   └── styles.css
│   ├── templates/
│   │   ├── core/
│   │   │   └── index.html
│   │   ├── _base.html
│   │   └── navigation.html
│   └── __init__.py
├── .env
├── config.py
└── manage.py

How to Implement User Registration

First of all, we'll create a registration form using Flask-WTF. Create a forms.py file inside the accounts package and add the following code:

from flask_wtf import FlaskForm
from wtforms import EmailField, PasswordField
from wtforms.validators import DataRequired, Email, EqualTo, Length

from src.accounts.models import User


class RegisterForm(FlaskForm):
    username = StringField(
        "Username", validators=[DataRequired(), Length(min=6, max=40)]
    )
    password = PasswordField(
        "Password", validators=[DataRequired(), Length(min=6, max=25)]
    )
    confirm = PasswordField(
        "Repeat password",
        validators=[
            DataRequired(),
            EqualTo("password", message="Passwords must match."),
        ],
    )

    def validate(self, extra_validators):
        initial_validation = super(RegisterForm, self).validate(extra_validators)
        if not initial_validation:
            return False
        user = User.query.filter_by(username=self.username.data).first()
        if user:
            self.username.errors.append("Username already registered")
            return False
        if self.password.data != self.confirm.data:
            self.password.errors.append("Passwords must match")
            return False
        return True

The RegisterForm extends the FlaskForm class and contains three fields – username, password, and confirm. We have added different validators such as DataRequired, Length, Email, and EqualTo to the respective fields.

We also defined a validate() method that is automatically called when the form is submitted.

Inside the method, we first perform the initial validation provided by FlaskForm. If that is successful, we perform our custom validation such as checking whether user is already registered and matching the password with the confirmed password. If there are any errors, we append the error message in the respective fields.

Now, let's use this form inside the HTML file. Create an accounts directory inside the templates folder and add a new file called register.html inside it. Add the following code:

{% extends "_base.html" %}

{% block content %}

<div class="row">
  <div class="col-md-4"></div>
  <div class="col-md-4">
    <main class="form-signin w-100 m-auto">
      <form role="form" method="post" action="">
        {{ form.csrf_token }}
        <h1 class="h3 mb-3 fw-normal text-center">Please register</h1>

        <div class="form-floating">
          {{ form.username(placeholder="username", class="form-control mb-2") }}
          {{ form.username.label }}
            {% if form.username.errors %}
              {% for error in form.username.errors %}
                <div class="alert alert-danger" role="alert">
                  {{ error }}
                </div>
              {% endfor %}
            {% endif %}
        </div>
        <div class="form-floating">
          {{ form.password(placeholder="password", class="form-control mb-2") }}
          {{ form.password.label }}
            {% if form.password.errors %}
              {% for error in form.password.errors %}
                <div class="alert alert-danger" role="alert">
                  {{ error }}
                </div>
              {% endfor %}
            {% endif %}
        </div>
        <div class="form-floating">
          {{ form.confirm(placeholder="Confirm Password", class="form-control mb-2") }}
          {{ form.confirm.label }}
            {% if form.confirm.errors %}
              {% for error in form.confirm.errors %}
                <div class="alert alert-danger" role="alert">
                  {{ error }}
                </div>
              {% endfor %}
            {% endif %}
        </div>
        <button class="w-100 btn btn-lg btn-primary" type="submit">Sign up</button>
        <p class="text-center mt-3">Already registered? <a href="{{ url_for('accounts.login') }}">Login now</a></p>
      </form>
    </main>
  </div>
  <div class="col-md-4"></div>
</div>

{% endblock %}

In the above Jinja template, we make use of the form that we created and add relevant error-handling logic checks for validation errors in each field. Users can submit the form by clicking the "Sign up" button, and a link below the form allows already registered users to navigate to the login page for authentication.

Next, let's use this form in the views.py to create a function to handle the registration process.

from .forms import RegisterForm
from src.accounts.models import User
from src import db, bcrypt
from flask_login import current_user
from flask import Blueprint, flash, redirect, render_template, request, url_for

accounts_bp = Blueprint("accounts", __name__)

HOME_URL = "core.home"
SETUP_2FA_URL = "accounts.setup_two_factor_auth"
VERIFY_2FA_URL = "accounts.verify_two_factor_auth"

@accounts_bp.route("/register", methods=["GET", "POST"])
def register():
    if current_user.is_authenticated:
        if current_user.is_two_factor_authentication_enabled:
            flash("You are already registered.", "info")
            return redirect(url_for(HOME_URL))
        else:
            flash("You have not enabled 2-Factor Authentication. Please enable first to login.", "info")
            return redirect(url_for(SETUP_2FA_URL))
    form = RegisterForm(request.form)
    if form.validate_on_submit():
        try:
            user = User(username=form.username.data, password=form.password.data)
            db.session.add(user)
            db.session.commit()

            login_user(user)
            flash("You are registered. You have to enable 2-Factor Authentication first to login.", "success")

            return redirect(url_for(SETUP_2FA_URL))
        except Exception:
            db.session.rollback()
            flash("Registration failed. Please try again.", "danger")

    return render_template("accounts/register.html", form=form)

The route begins by checking if the current user is already authenticated. If so, it verifies whether 2FA is enabled for the user. If 2FA is already enabled, a message informs the user that they're already registered, redirecting them to the home URL. However, if the user is authenticated but 2FA is not enabled, a flash message prompts the user to enable 2FA first before logging in, redirecting them to the 2FA setup URL.

If the user is not authenticated or has not yet registered 2FA, the code initializes a registration form and proceeds to validate the form data on submission. Upon successful form validation, we create a new User object with the provided username and password and save it to the database.

Upon successful user registration, the newly registered user is logged in. A success message flashes, notifying the user of successful registration and prompting them to enable 2FA before logging in. Subsequently, the user is redirected to the 2FA setup URL to enable 2FA.

How to Implement User Login

First, let's create a login form in the accounts/forms.py file:

class LoginForm(FlaskForm):
    username = StringField("Username", validators=[DataRequired()])
    password = PasswordField("Password", validators=[DataRequired()])

The form is similar to the registration form but it has only two fields – username and password.

Now, let's use this form inside a new HTML file called login.html created inside the templates/accounts directory. Add the following code:

{% extends "_base.html" %}

{% block content %}

<div class="row">
  <div class="col-md-4"></div>
  <div class="col-md-4">
    <main class="form-signin w-100 m-auto">
      <form role="form" method="post" action="">
        {{ form.csrf_token }}
        <h1 class="h3 mb-3 fw-normal text-center">Please sign in</h1>

        <div class="form-floating">
          {{ form.username(placeholder="username", class="form-control mb-2") }}
          {{ form.username.label }}
            {% if form.username.errors %}
              {% for error in form.username.errors %}
              <div class="alert alert-danger" role="alert">
                {{ error }}
              </div>
              {% endfor %}
            {% endif %}
        </div>
        <div class="form-floating">
          {{ form.password(placeholder="password", class="form-control mb-2") }}
          {{ form.password.label }}
            {% if form.password.errors %}
              {% for error in form.password.errors %}
                <div class="alert alert-danger" role="alert">
                  {{ error }}
                </div>
              {% endfor %}
            {% endif %}
        </div>
        <button class="w-100 btn btn-lg btn-primary" type="submit">Sign in</button>
        <p class="text-center mt-3">New User? <a href="{{ url_for('accounts.register') }}">Register now</a></p>
      </form>
    </main>
  </div>
  <div class="col-md-4"></div>
</div>

{% endblock %}

The above HTML file is also similar to the register.html file but with just two fields for the username and password.

Next, let's create a view function to handle the login process inside the accounts/views.py file:

from .forms import LoginForm, RegisterForm

@accounts_bp.route("/login", methods=["GET", "POST"])
def login():
    if current_user.is_authenticated:
        if current_user.is_two_factor_authentication_enabled:
            flash("You are already logged in.", "info")
            return redirect(url_for(HOME_URL))
        else:
            flash("You have not enabled 2-Factor Authentication. Please enable first to login.", "info")
            return redirect(url_for(SETUP_2FA_URL))

    form = LoginForm(request.form)
    if form.validate_on_submit():
        user = User.query.filter_by(username=form.username.data).first()
        if user and bcrypt.check_password_hash(user.password, request.form["password"]):
            login_user(user)
            if not current_user.is_two_factor_authentication_enabled:
                flash(
                    "You have not enabled 2-Factor Authentication. Please enable first to login.", "info")
                return redirect(url_for(SETUP_2FA_URL))
            return redirect(url_for(VERIFY_2FA_URL))
        elif not user:
            flash("You are not registered. Please register.", "danger")
        else:
            flash("Invalid username and/or password.", "danger")
    return render_template("accounts/login.html", form=form)

The route starts by checking if the current user is already authenticated. If the user is authenticated and 2FA is enabled, a message informs the user they're already logged in, redirecting them to the home URL. If the user is authenticated but 2FA isn't enabled, a flash message prompts the user to enable 2FA before logging in, redirecting them to the 2FA setup URL.

If the user isn't authenticated, the code initializes a login form and validates the form data upon submission. Upon successful validation, it queries the database to find a user matching the provided username. If the user exists and the password matches the hashed password stored in the database, the user is logged in.

Additionally, if 2FA isn't enabled for the current user after successful login, a flash message prompts the user to enable 2FA before proceeding, redirecting them to the 2FA setup URL. If the login is successful and 2FA is enabled, the user is redirected to the 2FA verification URL.

If the user isn't registered, a flash message informs them to register. If there's a mismatch in the provided username or password, another flash message notifies the user of invalid credentials.

How to Log Out the Users

Logging out the user is a very simple process. You just need to create a view function for it inside the accounts/views.py file:

from flask_login import login_required, login_user, logout_user


@accounts_bp.route("/logout")
@login_required
def logout():
    logout_user()
    flash("You were logged out.", "success")
    return redirect(url_for("accounts.login"))

The Flask-Login library contains a logout_user method that removes the user from the session. We used the @login_required decorator so that only authenticated users could logout.

How to Add the Setup 2FA Page

Up until now, we have been redirecting the users to the setup 2FA page whenever the 2FA is not enabled in their accounts, but we haven't implemented it yet. Let's do that in this section.

Let's start with the route for the page:

from src.utils import get_b64encoded_qr_image

@accounts_bp.route("/setup-2fa")
@login_required
def setup_two_factor_auth():
    secret = current_user.secret_token
    uri = current_user.get_authentication_setup_uri()
    base64_qr_image = get_b64encoded_qr_image(uri)
    return render_template("accounts/setup-2fa.html", secret=secret, qr_image=base64_qr_image)

The route, created inside accounts/views.py, ensures that only authenticated users can access it using the @login_required decorator.

Upon accessing this route, the function retrieves the current user's secret_token for 2FA setup and generates a URI through current_user.get_authentication_setup_uri() to configure an authenticator app like Google Authenticator.

It also uses get_b64encoded_qr_image(uri) to obtain a Base64-encoded QR code image representing this setup URI. We will define it below.

Finally, it renders the setup-2fa.html template, passing the user's secret_token and the Base64-encoded QR image to the template for users to scan it.

Next, create a utils.py file in the src directory and add the following code to generate the QR:

from io import BytesIO
import qrcode
from base64 import b64encode


def get_b64encoded_qr_image(data):
    print(data)
    qr = qrcode.QRCode(version=1, box_size=10, border=5)
    qr.add_data(data)
    qr.make(fit=True)
    img = qr.make_image(fill_color='black', back_color='white')
    buffered = BytesIO()
    img.save(buffered)
    return b64encode(buffered.getvalue()).decode("utf-8")