What is a randomized algorithm?

Randomized algorithms are algorithms that use random numbers to determine what to do next at any point in their logic. This means that it uses randomization for operation in the logic. Due to this randomness, the time and space complexity of the standard algorithm is reduced.

In other words, randomized algorithms use random numbers to find solutions to problems and improve the performance of the algorithms. Such algorithms are categorized into the following two types:

1. Monte Carlo randomized algorithm: These randomized algorithms have a small probability that their output is incorrect.
2. Las Vegas algorithm: These randomized algorithms always give an accurate result or alert the user about its failure.

Approximation algorithm

An approximation algorithm finds approximate solutions for optimization problems. They do not always guarantee to produce the best solution but they try to come close to the optimal value in polynomial time. Approximation algorithms guarantee to provide highly accurate solutions for optimization problems.

Randomized approximation algorithm

Approximation algorithms are efficient algorithms for solving optimization problems. However, when the concept of randomness is used in approximation algorithms, it enhances the algorithm’s performance.

Randomized approximation algorithms efficiently solve problems for which no deterministic algorithm has been identified. They are also used in the simulated annealing methods for solving optimization problems.

Randomized algorithms do random selections that vary depending on the input given to an algorithm to solve a problem. If a significant part of deterministic algorithms works well for the given input, a procedure to restart the algorithm after a particular time during its execution is determined, which helps speed up the algorithms.

Randomized approximation algorithm example (Max-Cut algorithm)

It is possible to prove that the max-cut problem produces 2-approximation using the greedy approximation algorithm. However, if randomization techniques are used in the max-cut problem, a shorter algorithm that gives the same performance and executes in polynomial time can be determined.

Random max-cut problem: Consider an undirected graph G = (V, E) and positive weight for every edge, select a random cut (partition V) in two subsets A, B such that it maximizes the combined weight of edges with one endpoint in A and the other in B. If less than m/2 edges are intersecting within the cut, repeat the process.

Theorem: A random-cut is a ½ approximation algorithm for a max-cut algorithm that executes in expected polynomial time.

Proof:

Consider X as a random variable representing the number of edges intersecting in the random cut. For i = 1, 2,..., m, X_i will be the indicator variable. If the i^th edge intersects the random cut, the value of the indicator variable will be 1. Otherwise, it will be 0. So, X = $\underset{i=1}{\overset{m}{\sum X_i}}$ and by the linearity of expectation $\underset{i = 1}{\overset{m}{E\left[X\right] =\sum E\left[X_i\right]}}$. Now, the probability for an edge {u, v} to lie in a randomly selected cut is ½, as the edges u and v will be placed in either of the two partitions. Hence, the chances of u and v being in one partition will be ½. Therefore, E[X_i] = ½ for every i. Hence, E[Xi] = m/2.

This proves that the random cut will give at least m/2 crossing edges. Now, the goal is to find a randomized algorithm that generates a good cut every time, and its execution time is a random variable with a polynomial-time execution. This can be done by computing the probability that $X \ge m/2$  when a random cut is selected. In the worst case, if$X \ge m/2$, all the probability is weighted on m, and if X < m/2, all the probability is weighted on m/2 - 1. This makes the expectation of X the highest.

$m/2 = E[X] \le (1 - P r[X \ge m/2] (m/2 - 1) + P r[X \ge m/2]m.$

By solving $P r[X \ge m/2]$, the solution will be at least 2/(m+2). This means the maximum number of total iterations in the algorithm will be (m+2) / 2. Hence proved that the algorithm executes within expected polynomial time and generates a random cut of size at m/2. In other terms, it can be marked that the expected weight of the generated cut is at least half the weight of the maximum cut.

Application of randomized approximation algorithm

One of the main applications of random approximation algorithms is randomized backtracking. Backtracking helps in solving combinatorial problems and it typically takes exponential time for execution. However, recent research proved that when the concept of randomness is involved in backtracking problems, they can be solved much quicker.

Backtracking problems typically contain small substructures in them. Each substructure has a property that the solution for an entire problem can be identified if anyone substructure is solved correctly. Due to this reason, these problems become tenable when solved using randomization techniques. The overall execution time that backtracking takes to generate a solution is reduced impressively by constantly restarting the backtracking technique after a polynomial time.

Derandomization

The methodology of removing randomness from a problem is called derandomization. It is not proved that all algorithms can be derandomized without increasing their execution time. However, the technique is used for removing randomization from several randomized approximation algorithms.

Derandomization of algorithms can be done using the following techniques:

• The method of conditional probabilities converts a proof of the algorithm into an efficient deterministic algorithm. This means it derandomizes the proof.
• Discrepancy theory helps in derandomizing geometric algorithms. It expresses the deviation of a particular situation from the start that one would expect it to be in.
• Using expander graphs to increase a limited portion of initial randomness.
• Transforming the randomized algorithm for using a hash function as the source of the algorithm’s randomness. Then, the algorithm is derandomized using brute-force techniques. This technique searches the sample space to make the algorithm deterministic.

Context and Applications

The topic randomized approximation algorithm is an essential topic in the theory of computation. The concept is studied in graduate and postgraduate courses like:

• Bachelors in Computer Science
• Masters in Computer Science
• Masters in Artificial Intelligence

Practice Problems

Q1. Which of these is a type of randomized algorithm?

1. Las Vegas
2. Polynomial time
3. Hashing
4. Minimum cut

Answer: Option a

Explanation: Las Vegas and Monte Carlo algorithms are the two types of randomized algorithms. This classification is done based on the output they produce.

Q2. Which randomized algorithm alerts the user in the case of failure?

1. Cambridge university press
2. Las Vegas
3. Hashing polynomial time
4. Maximum cut randomized algorithms

Answer: Option b

Explanation: The Las Vegas randomized algorithms provide the correct output. However, if the algorithm fails, it alerts the user. 

Q3. Is it possible to use the concept of randomization in approximation algorithms?

1. Up to a certain extent in quicksort
2. No
3. Yes
4. Depends on the problem

Answer: Option c

Explanation: The concept of randomization is used in approximation algorithms to boost the performance of the algorithms. Such algorithms are randomized approximation algorithms.

Q4. Randomized approximation algorithms are used to solve which type of problems?

1. In which no deterministic algorithm is defined
2. Hashing polynomial time randomized algorithms
3. Cambridge university press randomized algorithms
4. Randomized algorithms quicksort

Answer: Option a

Explanation: Randomized approximation algorithms are used for efficiently solving problems for which no deterministic algorithm is known.

Q5. Which of these is a derandomization technique?

1. Discrepancy theory
2. Quicksort randomized algorithms
3. Polynomial time randomized algorithms
4. Minimum cut randomized algorithms

Answer: Option a

Explanation: Discrepancy theory is a technique used for derandomizing algorithms. The theory is generally used to derandomize geometric algorithms. 

Common Mistakes

Students often use the terms randomized algorithms and randomized approximation algorithms interchangeably. However, they are different concepts. Hence, students should not get confused while using these terms.

Related Concepts

• Randomized quicksort
• Minimum cut problem
• Approximation algorithm
• Convex programming
• Greedy approximation algorithm
• Randomized algorithms