Jul 21 16:24:46.844102 ip-10-0-132-151 systemd[1]: Starting Kubernetes Kubelet... Jul 21 16:24:47.252009 ip-10-0-132-151 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 16:24:47.252009 ip-10-0-132-151 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 16:24:47.252009 ip-10-0-132-151 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 16:24:47.252009 ip-10-0-132-151 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 16:24:47.252009 ip-10-0-132-151 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 16:24:47.252009 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.248923 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254165 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254179 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254183 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254186 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254190 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254192 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254202 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254205 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254208 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254210 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254213 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254216 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254220 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254222 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254225 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254228 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254230 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254233 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 16:24:47.268061 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254236 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254239 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254242 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254245 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254247 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254250 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254253 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254256 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254258 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254261 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254264 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254266 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254269 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254271 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254274 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254276 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254279 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254281 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254284 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254286 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 16:24:47.269405 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254289 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254292 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254295 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254298 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254301 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254304 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254306 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254309 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254311 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254314 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254317 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254319 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254322 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254326 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254328 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254331 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254338 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254340 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254343 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254346 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 16:24:47.271783 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254348 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254351 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254354 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254356 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254358 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254361 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254363 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254366 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254369 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254371 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254373 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254376 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254380 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254382 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254385 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254389 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254393 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254395 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254398 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254401 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 16:24:47.273141 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254403 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254406 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254408 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254411 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254413 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254416 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254419 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254423 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254427 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254430 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254433 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254436 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254439 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254442 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254445 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254448 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254450 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254453 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254455 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 16:24:47.275562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254458 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.254460 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255167 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255174 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255177 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255180 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255185 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255189 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255192 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255195 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255198 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255201 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255204 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255206 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255210 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255213 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255216 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255219 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255223 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255226 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 16:24:47.276982 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255229 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255232 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255235 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255237 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255240 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255242 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255245 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255248 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255250 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255253 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255256 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255258 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255261 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255263 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255266 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255268 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255271 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255273 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255275 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255278 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 16:24:47.279698 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255281 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255284 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255286 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255289 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255291 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255294 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255296 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255299 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255301 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255303 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255306 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255309 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255311 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255314 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255317 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255319 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255322 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255325 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255327 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255330 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 16:24:47.309352 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255332 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255335 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255337 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255340 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255342 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255344 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255347 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255349 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255352 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255354 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255356 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255362 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255365 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255368 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255371 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255375 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255379 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255382 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255385 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 16:24:47.310941 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255387 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255390 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255393 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255396 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255398 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255401 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255404 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255406 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255409 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255411 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255414 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255416 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255419 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255421 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255424 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255426 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255429 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255431 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255433 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255436 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 16:24:47.313049 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255438 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255440 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255849 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255855 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255860 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255864 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255867 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255870 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255873 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255876 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255878 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255881 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255884 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255887 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255891 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255895 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255898 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255901 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255903 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255906 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 16:24:47.329796 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255909 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 16:24:47.326875 ip-10-0-132-151 systemd[1]: Started Kubernetes Kubelet. Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255912 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255915 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255918 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255920 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255923 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255926 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255928 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255931 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255933 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255936 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255938 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255941 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255943 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255945 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255948 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255952 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255954 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255957 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255960 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 16:24:47.331489 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255962 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255965 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255968 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255970 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255973 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255975 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255978 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255981 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255984 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255987 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255990 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255994 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.255997 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256000 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256002 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256005 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256008 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256011 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256014 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 16:24:47.333151 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256016 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256019 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256022 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256025 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256028 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256030 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256033 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256035 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256038 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256041 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256044 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256047 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256049 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256052 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256055 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256057 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256060 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256063 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256065 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 16:24:47.349624 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256068 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256071 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256075 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256077 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256080 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256082 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256085 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256087 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256089 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256092 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256097 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256099 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256102 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256104 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256107 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256109 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256112 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256114 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256117 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 16:24:47.352580 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256119 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256122 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256124 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256128 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256197 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256204 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256212 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256219 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256226 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256230 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256234 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256238 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256241 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256244 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256248 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256251 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256254 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256257 2557 flags.go:64] FLAG: --cgroup-root="" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256260 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256262 2557 flags.go:64] FLAG: --client-ca-file="" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256265 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256268 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256272 2557 flags.go:64] FLAG: --cluster-domain="" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256276 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 16:24:47.355373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256279 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256282 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256286 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256290 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256292 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256296 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256299 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256302 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256305 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256308 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256311 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256315 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256319 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256322 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256326 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256328 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256331 2557 flags.go:64] FLAG: --enable-server="true" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256343 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256347 2557 flags.go:64] FLAG: --event-burst="100" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256350 2557 flags.go:64] FLAG: --event-qps="50" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256353 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256356 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256359 2557 flags.go:64] FLAG: --eviction-hard="" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256367 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 16:24:47.356632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256370 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256373 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256376 2557 flags.go:64] FLAG: --eviction-soft="" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256379 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256382 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256385 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256387 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256393 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256396 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256399 2557 flags.go:64] FLAG: --feature-gates="" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256402 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256405 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256408 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256411 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256414 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256417 2557 flags.go:64] FLAG: --help="false" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256420 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256423 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256425 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256429 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256433 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256436 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256439 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 16:24:47.371413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256442 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256445 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256448 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256451 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256454 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256457 2557 flags.go:64] FLAG: --kube-reserved="" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256460 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256462 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256465 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256468 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256471 2557 flags.go:64] FLAG: --lock-file="" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256474 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256477 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256480 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256485 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256488 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256491 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256494 2557 flags.go:64] FLAG: --logging-format="text" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256497 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256501 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256503 2557 flags.go:64] FLAG: --manifest-url="" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256507 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256511 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256514 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256518 2557 flags.go:64] FLAG: --max-pods="110" Jul 21 16:24:47.386873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256521 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256524 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256527 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256531 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256534 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256537 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256540 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256546 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256549 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256552 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256555 2557 flags.go:64] FLAG: --pod-cidr="" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256559 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256563 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256566 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256569 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256572 2557 flags.go:64] FLAG: --port="10250" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256575 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256578 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08d90d13208fdcbf0" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256581 2557 flags.go:64] FLAG: --qos-reserved="" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256592 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256596 2557 flags.go:64] FLAG: --register-node="true" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256599 2557 flags.go:64] FLAG: --register-with-taints="" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256602 2557 flags.go:64] FLAG: --registry-burst="10" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256607 2557 flags.go:64] FLAG: --registry-qps="5" Jul 21 16:24:47.528625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256610 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256612 2557 flags.go:64] FLAG: --reserved-memory="" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256616 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256619 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256622 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256625 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256628 2557 flags.go:64] FLAG: --runonce="false" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256630 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256633 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256636 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256639 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256642 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256646 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256650 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256652 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256655 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256658 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256661 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256663 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256666 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256670 2557 flags.go:64] FLAG: --system-cgroups="" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256672 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256678 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256681 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256684 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 16:24:47.546371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256688 2557 flags.go:64] FLAG: --tls-min-version="" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256690 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256693 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256696 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256699 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256701 2557 flags.go:64] FLAG: --v="2" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256706 2557 flags.go:64] FLAG: --version="false" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256710 2557 flags.go:64] FLAG: --vmodule="" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256713 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.256716 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256852 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256857 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256860 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256863 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256866 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256869 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256871 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256874 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256876 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256880 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256882 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 16:24:47.547393 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256885 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256887 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256890 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256892 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256895 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256897 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256900 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256903 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256905 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256908 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256910 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256913 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256915 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256918 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256920 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256923 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256925 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256929 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256932 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 16:24:47.548202 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256934 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256938 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256942 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256945 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256947 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256950 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256953 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256955 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256958 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256961 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256963 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256966 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256969 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256973 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256976 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256979 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256982 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256984 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256987 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 16:24:47.549197 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256989 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256992 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256995 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.256997 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257000 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257003 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257005 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257007 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257010 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257013 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257015 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257018 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257021 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257023 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257026 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257028 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257031 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257033 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257035 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257038 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 16:24:47.550329 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257040 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257043 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257045 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257048 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257051 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257054 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257057 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257059 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257062 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257064 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257066 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257069 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257071 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257073 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257076 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257079 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257081 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257084 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257086 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257089 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 16:24:47.551330 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257093 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257096 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257099 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257102 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257105 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257108 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257110 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257112 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257115 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.257117 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.257692 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.265934 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.265949 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.265969 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.265975 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 16:24:47.551976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266031 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266037 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266040 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266043 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266046 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266049 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266052 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266055 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266057 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266060 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266062 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266065 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266068 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266072 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266074 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266077 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266080 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266082 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266085 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266087 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 16:24:47.552787 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266091 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266095 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266100 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266103 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266105 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266108 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266110 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266113 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266116 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266118 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266121 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266123 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266126 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266128 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266131 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266133 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266136 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266138 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266141 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266143 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 16:24:47.554045 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266146 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266148 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266153 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266157 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266160 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266163 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266169 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266172 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266175 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266178 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266181 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266183 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266186 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266189 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266192 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266195 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266198 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266200 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266203 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 16:24:47.555595 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266206 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266209 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266213 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266216 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266219 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266222 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266225 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266228 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266230 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266233 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266235 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266238 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266241 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266243 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266246 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266248 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266253 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266255 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266258 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266261 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 16:24:47.561548 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266265 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266268 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266271 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266273 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266276 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266279 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266281 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266284 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266286 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266289 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266291 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266294 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266297 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266299 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266302 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266304 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266307 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266309 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266312 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 16:24:47.563445 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266314 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.266319 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266429 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266434 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266437 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266439 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266442 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266446 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266450 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266454 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266457 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266460 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266463 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266467 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266470 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 16:24:47.565284 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266474 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266476 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266479 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266481 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266484 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266486 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266489 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266492 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266494 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266497 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266499 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266502 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266504 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266507 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266509 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266512 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266515 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266517 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266520 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266522 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 16:24:47.567593 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266525 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266527 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266530 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266532 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266535 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266537 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266541 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266544 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266546 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266549 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266553 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266555 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266559 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266561 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266564 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266566 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266569 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266571 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266574 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266577 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 16:24:47.568778 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266579 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266581 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266584 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266586 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266589 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266591 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266594 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266597 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266599 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266602 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266604 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266607 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266609 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266612 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266615 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266617 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266620 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266623 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266628 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266631 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 16:24:47.569529 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266634 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266636 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266641 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266643 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266646 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266649 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266652 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266655 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266658 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266660 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266663 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266665 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266668 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266671 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266673 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266676 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266678 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266681 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266683 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266687 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 16:24:47.570122 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266690 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266693 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266696 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266698 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266701 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:47.266703 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.266708 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.266814 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.268870 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.270210 2557 server.go:1019] "Starting client certificate rotation" Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.270321 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.270347 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.273091 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 16:24:47.776064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.274945 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 16:24:47.776947 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.291836 2557 log.go:25] "Validated CRI v1 runtime API" Jul 21 16:24:47.776947 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.297491 2557 log.go:25] "Validated CRI v1 image API" Jul 21 16:24:47.776947 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.298252 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 16:24:47.776947 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.299040 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 16:24:47.776947 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.301291 2557 fs.go:135] Filesystem UUIDs: map[32c6de9b-1a8d-4e38-841e-ff0efe3c01ab:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 93dc211c-1da2-4972-975a-20d2b0cf5afb:/dev/nvme0n1p3] Jul 21 16:24:47.776947 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.301305 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.306179 2557 manager.go:216] Machine: {Timestamp:2026-07-21 16:24:47.304911012 +0000 UTC m=+0.343746665 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097260 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d55a527f137538772fa827b87a3e4 SystemUUID:ec2d55a5-27f1-3753-8772-fa827b87a3e4 BootID:d4df0d23-cba2-4118-a310-bdaf4df1b263 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:73:1f:ae:86:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:73:1f:ae:86:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:8b:8d:62:e5:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.306274 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.306352 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.309569 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.309590 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-151.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.310274 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.310283 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.310290 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.311831 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.313110 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.315867 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.315978 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.316004 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.316013 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.316026 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.316036 2557 kubelet.go:398] "Adding apiserver pod source" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.316047 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 16:24:47.777393 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.316344 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.316362 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.318793 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.320695 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.320723 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321777 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321790 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321796 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321803 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321812 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321831 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321843 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321849 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321856 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321861 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321874 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.321880 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.322591 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.322599 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.326006 2557 server.go:1268] "Started kubelet" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.326048 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.326160 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.326240 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.327718 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.328041 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-151.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.328371 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.329459 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.329460 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.333041 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c45ab55c10da61 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-21 16:24:47.325747809 +0000 UTC m=+0.364583462,LastTimestamp:2026-07-21 16:24:47.325747809 +0000 UTC m=+0.364583462,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.335647 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.336244 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.336258 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.336334 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.336761 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.336764 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.336794 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.336890 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.336911 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.337267 2557 factory.go:55] Registering systemd factory Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.337282 2557 factory.go:223] Registration of the systemd container factory successfully Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.339523 2557 factory.go:153] Registering CRI-O factory Jul 21 16:24:47.778213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.339547 2557 factory.go:223] Registration of the crio container factory successfully Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.339604 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.339638 2557 factory.go:103] Registering Raw factory Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.339654 2557 manager.go:1196] Started watching for new ooms in manager Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.340075 2557 manager.go:349] Starting recovery of all containers Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.341685 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.341788 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.348741 2557 manager.go:354] Recovery completed Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.349982 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v4kfh" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.353107 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.355306 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.355338 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.355350 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.355878 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.355885 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.355887 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.355957 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.357337 2557 policy_none.go:49] "None policy: Start" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.357351 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.357364 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.357840 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.357861 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.357873 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.357810 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c45ab55dd41bc6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-151.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-21 16:24:47.355321286 +0000 UTC m=+0.394156946,LastTimestamp:2026-07-21 16:24:47.355321286 +0000 UTC m=+0.394156946,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.358144 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v4kfh" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.358962 2557 policy_none.go:47] "Start" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.406475 2557 manager.go:337] "Starting Device Plugin manager" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.406666 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.406676 2557 server.go:87] "Starting device plugin registration server" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.406903 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.406913 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.407099 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.407224 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.407234 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.407860 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.407912 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.481040 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.482229 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.482268 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 16:24:47.779384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.482296 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.482337 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.484292 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.507144 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.508043 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.508079 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.508090 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.508111 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.516984 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.517003 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-151.ec2.internal\": node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.544904 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.582544 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal"] Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.582658 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.583969 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.584002 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.584014 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586084 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586186 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586214 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586754 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586781 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586791 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586880 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586903 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.586913 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.588724 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.588746 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.589638 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.589657 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.589669 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.602376 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-151.ec2.internal\" not found" node="ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.606134 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-151.ec2.internal\" not found" node="ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.638510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90ea7be93df65d66da652d9417af881e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"90ea7be93df65d66da652d9417af881e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.780561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.638530 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.638548 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.645599 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.739579 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90ea7be93df65d66da652d9417af881e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"90ea7be93df65d66da652d9417af881e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.739525 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90ea7be93df65d66da652d9417af881e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"90ea7be93df65d66da652d9417af881e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.739646 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.739662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.739689 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.739710 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.781511 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.746623 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:47.945175 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.847095 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:47.945175 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.905299 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 21 16:24:47.945175 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:47.908610 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:48.127272 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:47.947843 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.127272 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:48.048544 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.308838 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:48.149316 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.308838 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:48.249996 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.308838 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.269870 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 16:24:48.308838 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.270005 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 16:24:48.467896 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:48.350776 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.467896 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.366308 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 16:19:48 +0000 UTC" deadline="2028-02-21 16:41:14.911682489 +0000 UTC" Jul 21 16:24:48.467896 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.366340 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13920h16m26.545345819s" Jul 21 16:24:48.467896 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:48.451855 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.552417 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:48.552382 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.653325 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:48.653289 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 21 16:24:48.706114 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:48.706078 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fcf4b6dde762c7ce3ec0cc2a1506cd.slice/crio-5b6ef1b12c2093377fa226d31d57610c364e079a0a96de5cdbc26cfe5b1651a0 WatchSource:0}: Error finding container 5b6ef1b12c2093377fa226d31d57610c364e079a0a96de5cdbc26cfe5b1651a0: Status 404 returned error can't find the container with id 5b6ef1b12c2093377fa226d31d57610c364e079a0a96de5cdbc26cfe5b1651a0 Jul 21 16:24:48.708448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.708412 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 16:24:48.713542 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.713524 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 16:24:48.718081 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.718030 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 16:24:48.720631 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:48.720609 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ea7be93df65d66da652d9417af881e.slice/crio-4d3134d069f1ae6b6a7bba201d70ebcaf66a98f1fbfc3d0750e78a8acaebd86f WatchSource:0}: Error finding container 4d3134d069f1ae6b6a7bba201d70ebcaf66a98f1fbfc3d0750e78a8acaebd86f: Status 404 returned error can't find the container with id 4d3134d069f1ae6b6a7bba201d70ebcaf66a98f1fbfc3d0750e78a8acaebd86f Jul 21 16:24:48.736627 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.736596 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 21 16:24:48.747967 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.747947 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 16:24:48.749785 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.749773 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 21 16:24:48.763037 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.763017 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 16:24:48.906473 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:48.906441 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 16:24:49.317347 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.317312 2557 apiserver.go:52] "Watching apiserver" Jul 21 16:24:49.325346 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.325318 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 16:24:49.325812 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.325783 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-xdwg8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj","openshift-cluster-node-tuning-operator/tuned-clmbb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal","openshift-multus/multus-additional-cni-plugins-5z485","openshift-multus/network-metrics-daemon-bglxt","openshift-network-diagnostics/network-check-target-6r7qk","openshift-ovn-kubernetes/ovnkube-node-z889p","kube-system/global-pull-secret-syncer-9t8kx","kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal","openshift-dns/node-resolver-dwl4v","openshift-image-registry/node-ca-js8z5","openshift-multus/multus-jbg76","openshift-network-operator/iptables-alerter-9lgj5"] Jul 21 16:24:49.328518 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.328498 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.331797 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.331227 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.332295 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.332104 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 16:24:49.332506 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.332102 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 16:24:49.332612 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.332132 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 16:24:49.332676 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.332186 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 16:24:49.332754 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.332272 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hcp6m\"" Jul 21 16:24:49.333644 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.333622 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.334685 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.334554 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 16:24:49.334823 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.334804 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 16:24:49.336115 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.335064 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 16:24:49.336115 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.335281 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qwc5p\"" Jul 21 16:24:49.336414 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.336393 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.337590 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.337570 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 16:24:49.338977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.338268 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:24:49.338977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.338373 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vtrrl\"" Jul 21 16:24:49.338977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.338752 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 16:24:49.340532 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.339980 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.341670 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.341648 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fqvxs\"" Jul 21 16:24:49.341770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.341659 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 16:24:49.341770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.341728 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 16:24:49.342810 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.342788 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 16:24:49.342937 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.342854 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 16:24:49.342937 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.342788 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zcsrq\"" Jul 21 16:24:49.343166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.343143 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:49.343273 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.343233 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:24:49.345270 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.345251 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:49.345350 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.345310 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:24:49.347882 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.347865 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.350373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.350355 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.350471 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.350428 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:24:49.350991 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.350972 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 16:24:49.351082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.350989 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 16:24:49.351082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351029 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 16:24:49.351193 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351085 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 16:24:49.351193 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351178 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dlrxd\"" Jul 21 16:24:49.351299 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351258 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 16:24:49.351299 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351283 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 16:24:49.351400 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351301 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 16:24:49.351797 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351772 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-cnibin\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.351877 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351815 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-k8s-cni-cncf-io\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.351877 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351843 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-etc-kubernetes\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352030 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351886 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-socket-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.352030 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351910 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-modprobe-d\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352030 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351929 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-system-cni-dir\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.352030 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351945 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cni-binary-copy\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.352030 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.351968 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.352030 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352028 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-cni-multus\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-hostroot\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352073 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-multus-certs\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352092 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhmmg\" (UniqueName: \"kubernetes.io/projected/ab204405-ee64-4517-9261-427c0a773d69-kube-api-access-qhmmg\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352108 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-os-release\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352134 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352163 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8h48c\" (UniqueName: \"kubernetes.io/projected/a34fa482-21e3-448c-ab10-2ca59ef86ae1-kube-api-access-8h48c\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352189 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-lib-modules\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352214 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352237 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-system-cni-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352261 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-cni-binary-copy\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352284 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-daemon-config\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352318 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-systemd\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352348 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-var-lib-kubelet\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352376 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-kubelet\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352419 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-kubernetes\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-sys\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab204405-ee64-4517-9261-427c0a773d69-etc-tuned\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5791f270-f605-4255-8fad-d441db9d0be3-agent-certs\") pod \"konnectivity-agent-xdwg8\" (UID: \"5791f270-f605-4255-8fad-d441db9d0be3\") " pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352556 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5791f270-f605-4255-8fad-d441db9d0be3-konnectivity-ca\") pod \"konnectivity-agent-xdwg8\" (UID: \"5791f270-f605-4255-8fad-d441db9d0be3\") " pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352594 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-netns\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352621 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-cni-bin\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352644 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-conf-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352673 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352702 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-etc-selinux\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysctl-conf\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352783 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-run\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-host\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-cni-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.352955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352900 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-os-release\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352922 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-socket-dir-parent\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352937 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352948 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-registration-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352967 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-device-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.352990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysconfig\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.353014 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab204405-ee64-4517-9261-427c0a773d69-tmp\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.353038 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cnibin\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.353064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgm27\" (UniqueName: \"kubernetes.io/projected/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-kube-api-access-wgm27\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.353087 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-sys-fs\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.353111 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysctl-d\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.353595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.353134 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf55c\" (UniqueName: \"kubernetes.io/projected/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-kube-api-access-jf55c\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.355568 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.355546 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 16:24:49.355661 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.355647 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.355935 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.355913 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 16:24:49.356175 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.356154 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bkf5d\"" Jul 21 16:24:49.356399 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.356383 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:24:49.358094 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.358076 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.360163 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.360147 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 16:24:49.361371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.361316 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 16:24:49.361371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.361328 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 16:24:49.361371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.361335 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 16:24:49.361572 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.361386 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mgvjg\"" Jul 21 16:24:49.361572 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.361386 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 16:24:49.361572 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.361479 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pwjt5\"" Jul 21 16:24:49.378470 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.378446 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-88zdf" Jul 21 16:24:49.384289 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.384267 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-88zdf" Jul 21 16:24:49.438937 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.438890 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 16:24:49.453942 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.453903 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-lib-modules\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.454082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.453955 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.454082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.453990 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-system-cni-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-cni-binary-copy\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-systemd\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.454082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-var-lib-kubelet\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454095 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-ovnkube-script-lib\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-kubelet\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454150 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-kubernetes\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454176 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab204405-ee64-4517-9261-427c0a773d69-etc-tuned\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5791f270-f605-4255-8fad-d441db9d0be3-agent-certs\") pod \"konnectivity-agent-xdwg8\" (UID: \"5791f270-f605-4255-8fad-d441db9d0be3\") " pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cni-binary-copy\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454250 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-netns\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-cni-bin\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454300 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-conf-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454325 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-etc-selinux\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454353 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysctl-conf\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454378 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-slash\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454404 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-etc-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454431 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-os-release\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454457 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-socket-dir-parent\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454483 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-registration-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysconfig\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cnibin\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454560 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-run-ovn-kubernetes\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454586 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-hosts-file\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454611 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-tmp-dir\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454643 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52e6d6c8-f788-4a30-b59a-81fc53754d7b-host-slash\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454671 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqknb\" (UniqueName: \"kubernetes.io/projected/52e6d6c8-f788-4a30-b59a-81fc53754d7b-kube-api-access-wqknb\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-k8s-cni-cncf-io\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-etc-kubernetes\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454749 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-system-cni-dir\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.454889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454777 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-cni-netd\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454803 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454839 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-cni-multus\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454865 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-hostroot\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454892 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-multus-certs\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454935 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454963 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-env-overrides\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.454987 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/44dab062-86b8-4849-9d19-75299742a580-dbus\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8h48c\" (UniqueName: \"kubernetes.io/projected/a34fa482-21e3-448c-ab10-2ca59ef86ae1-kube-api-access-8h48c\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455041 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52e6d6c8-f788-4a30-b59a-81fc53754d7b-iptables-alerter-script\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455125 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-cni-bin\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455159 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dda4441-e788-4fb1-8876-f736168d1c74-ovn-node-metrics-cert\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455183 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96c383c4-32ae-4252-bda3-6575b8d24b76-host\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455217 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-daemon-config\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455244 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-systemd\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.455767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455272 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-sys\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455298 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5791f270-f605-4255-8fad-d441db9d0be3-konnectivity-ca\") pod \"konnectivity-agent-xdwg8\" (UID: \"5791f270-f605-4255-8fad-d441db9d0be3\") " pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455324 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-run-netns\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455349 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96c383c4-32ae-4252-bda3-6575b8d24b76-serviceca\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455375 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xhps\" (UniqueName: \"kubernetes.io/projected/96c383c4-32ae-4252-bda3-6575b8d24b76-kube-api-access-7xhps\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455400 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-kubelet\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455432 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-run\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455483 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-host\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n7mj\" (UniqueName: \"kubernetes.io/projected/1dda4441-e788-4fb1-8876-f736168d1c74-kube-api-access-2n7mj\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455537 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-cni-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455566 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-device-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455592 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab204405-ee64-4517-9261-427c0a773d69-tmp\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455618 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-var-lib-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455652 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-ovnkube-config\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455680 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455708 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:49.456652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455735 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8nrh\" (UniqueName: \"kubernetes.io/projected/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-kube-api-access-p8nrh\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455766 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgm27\" (UniqueName: \"kubernetes.io/projected/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-kube-api-access-wgm27\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455803 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-sys-fs\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455836 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysctl-d\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455871 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf55c\" (UniqueName: \"kubernetes.io/projected/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-kube-api-access-jf55c\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-node-log\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455947 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/44dab062-86b8-4849-9d19-75299742a580-kubelet-config\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.455976 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-cnibin\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456004 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-socket-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456032 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-modprobe-d\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456107 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-systemd-units\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456136 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-ovn\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456164 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bncs\" (UniqueName: \"kubernetes.io/projected/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-kube-api-access-8bncs\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456194 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhmmg\" (UniqueName: \"kubernetes.io/projected/ab204405-ee64-4517-9261-427c0a773d69-kube-api-access-qhmmg\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-os-release\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-log-socket\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456469 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-lib-modules\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.456553 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-system-cni-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.457194 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-cni-binary-copy\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.457302 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-systemd\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.457363 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-var-lib-kubelet\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.457422 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-kubelet\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.457473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-kubernetes\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.457795 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.458196 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-k8s-cni-cncf-io\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.458283 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-etc-kubernetes\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.458335 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-system-cni-dir\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.458435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.458405 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-cni-multus\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.459001 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.458449 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-hostroot\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.459001 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.458495 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-multus-certs\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.459161 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.459108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-daemon-config\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.459218 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.459166 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.459274 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.459220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-sys\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.459789 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.459754 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-sys-fs\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.459955 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.459935 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysctl-d\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.460209 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.460190 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-host\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.460625 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.460606 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-cni-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.460715 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.460656 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-device-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.460777 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.460739 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5791f270-f605-4255-8fad-d441db9d0be3-konnectivity-ca\") pod \"konnectivity-agent-xdwg8\" (UID: \"5791f270-f605-4255-8fad-d441db9d0be3\") " pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.460877 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.460856 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.460954 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.460937 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-run\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.461114 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.461085 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.461231 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.461190 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-cnibin\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.461330 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.461312 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-socket-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.461386 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.461373 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-modprobe-d\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.461842 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.461811 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.461971 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.461951 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cni-binary-copy\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.462063 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462022 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-run-netns\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.462119 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462096 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-host-var-lib-cni-bin\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.462181 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462140 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-conf-dir\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.462181 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462157 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysconfig\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.462285 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462206 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-etc-selinux\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.462285 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462228 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-os-release\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.462402 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462299 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-multus-socket-dir-parent\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.462402 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462301 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ab204405-ee64-4517-9261-427c0a773d69-etc-sysctl-conf\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.462402 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462366 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a34fa482-21e3-448c-ab10-2ca59ef86ae1-registration-dir\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.462402 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462373 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-os-release\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.462620 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.462428 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-cnibin\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.463626 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.463605 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ab204405-ee64-4517-9261-427c0a773d69-etc-tuned\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.463764 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.463746 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab204405-ee64-4517-9261-427c0a773d69-tmp\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.466845 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.466810 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5791f270-f605-4255-8fad-d441db9d0be3-agent-certs\") pod \"konnectivity-agent-xdwg8\" (UID: \"5791f270-f605-4255-8fad-d441db9d0be3\") " pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.471154 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.471122 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8h48c\" (UniqueName: \"kubernetes.io/projected/a34fa482-21e3-448c-ab10-2ca59ef86ae1-kube-api-access-8h48c\") pod \"aws-ebs-csi-driver-node-pszhj\" (UID: \"a34fa482-21e3-448c-ab10-2ca59ef86ae1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.471567 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.471544 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf55c\" (UniqueName: \"kubernetes.io/projected/a59b6872-6c27-49c0-83b1-c6e3ecb3ed95-kube-api-access-jf55c\") pod \"multus-additional-cni-plugins-5z485\" (UID: \"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95\") " pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.471792 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.471769 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgm27\" (UniqueName: \"kubernetes.io/projected/ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be-kube-api-access-wgm27\") pod \"multus-jbg76\" (UID: \"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be\") " pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.476127 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.475999 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhmmg\" (UniqueName: \"kubernetes.io/projected/ab204405-ee64-4517-9261-427c0a773d69-kube-api-access-qhmmg\") pod \"tuned-clmbb\" (UID: \"ab204405-ee64-4517-9261-427c0a773d69\") " pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.498534 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.498487 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" event={"ID":"90ea7be93df65d66da652d9417af881e","Type":"ContainerStarted","Data":"4d3134d069f1ae6b6a7bba201d70ebcaf66a98f1fbfc3d0750e78a8acaebd86f"} Jul 21 16:24:49.500412 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.500386 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"15fcf4b6dde762c7ce3ec0cc2a1506cd","Type":"ContainerStarted","Data":"5b6ef1b12c2093377fa226d31d57610c364e079a0a96de5cdbc26cfe5b1651a0"} Jul 21 16:24:49.557159 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557122 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-cni-netd\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557361 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:49.557361 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557215 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-env-overrides\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557361 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557247 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/44dab062-86b8-4849-9d19-75299742a580-dbus\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.557361 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52e6d6c8-f788-4a30-b59a-81fc53754d7b-iptables-alerter-script\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.557361 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557300 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557361 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557324 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-cni-bin\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557361 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557349 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dda4441-e788-4fb1-8876-f736168d1c74-ovn-node-metrics-cert\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557373 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96c383c4-32ae-4252-bda3-6575b8d24b76-host\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-systemd\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557419 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-run-netns\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557442 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96c383c4-32ae-4252-bda3-6575b8d24b76-serviceca\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557468 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xhps\" (UniqueName: \"kubernetes.io/projected/96c383c4-32ae-4252-bda3-6575b8d24b76-kube-api-access-7xhps\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557494 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-kubelet\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557524 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n7mj\" (UniqueName: \"kubernetes.io/projected/1dda4441-e788-4fb1-8876-f736168d1c74-kube-api-access-2n7mj\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557554 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-var-lib-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557583 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-ovnkube-config\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557610 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557640 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8nrh\" (UniqueName: \"kubernetes.io/projected/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-kube-api-access-p8nrh\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557706 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-node-log\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.557735 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557732 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/44dab062-86b8-4849-9d19-75299742a580-kubelet-config\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557762 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-systemd-units\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557786 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-ovn\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557812 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bncs\" (UniqueName: \"kubernetes.io/projected/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-kube-api-access-8bncs\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557840 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-log-socket\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557869 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.557918 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-ovnkube-script-lib\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558040 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-slash\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558101 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-etc-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558135 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-kubelet\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558162 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-run-ovn-kubernetes\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558176 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/44dab062-86b8-4849-9d19-75299742a580-kubelet-config\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558189 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-hosts-file\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558216 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-tmp-dir\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558242 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52e6d6c8-f788-4a30-b59a-81fc53754d7b-host-slash\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558245 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-cni-netd\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqknb\" (UniqueName: \"kubernetes.io/projected/52e6d6c8-f788-4a30-b59a-81fc53754d7b-kube-api-access-wqknb\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.558448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558451 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-systemd-units\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558498 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-ovn\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558561 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-log-socket\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558585 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-var-lib-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558606 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-run-ovn-kubernetes\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-etc-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-hosts-file\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558676 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.558997 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-tmp-dir\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559056 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52e6d6c8-f788-4a30-b59a-81fc53754d7b-host-slash\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559092 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-run-netns\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559127 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96c383c4-32ae-4252-bda3-6575b8d24b76-host\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559162 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-systemd\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-env-overrides\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.559328 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559264 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-ovnkube-config\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559336 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1dda4441-e788-4fb1-8876-f736168d1c74-ovnkube-script-lib\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.559370 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559402 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-slash\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559472 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/44dab062-86b8-4849-9d19-75299742a580-dbus\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559498 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-host-cni-bin\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559508 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-run-openvswitch\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559499 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/52e6d6c8-f788-4a30-b59a-81fc53754d7b-iptables-alerter-script\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.559557 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret podName:44dab062-86b8-4849-9d19-75299742a580 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:50.059528405 +0000 UTC m=+3.098364067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret") pod "global-pull-secret-syncer-9t8kx" (UID: "44dab062-86b8-4849-9d19-75299742a580") : object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559603 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/96c383c4-32ae-4252-bda3-6575b8d24b76-serviceca\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.559612 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1dda4441-e788-4fb1-8876-f736168d1c74-node-log\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.559621 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:49.560048 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.559705 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs podName:64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd nodeName:}" failed. No retries permitted until 2026-07-21 16:24:50.059685088 +0000 UTC m=+3.098520728 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs") pod "network-metrics-daemon-bglxt" (UID: "64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:49.561542 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.561512 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1dda4441-e788-4fb1-8876-f736168d1c74-ovn-node-metrics-cert\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.569052 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.568905 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 16:24:49.569052 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.569013 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 16:24:49.569052 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.569028 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5d5kz for pod openshift-network-diagnostics/network-check-target-6r7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:49.569269 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:49.569089 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz podName:2af8f398-2b56-4964-a136-86ff864cbb35 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:50.069071526 +0000 UTC m=+3.107907174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5d5kz" (UniqueName: "kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz") pod "network-check-target-6r7qk" (UID: "2af8f398-2b56-4964-a136-86ff864cbb35") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:49.574145 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.574105 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8nrh\" (UniqueName: \"kubernetes.io/projected/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-kube-api-access-p8nrh\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:49.574600 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.574574 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n7mj\" (UniqueName: \"kubernetes.io/projected/1dda4441-e788-4fb1-8876-f736168d1c74-kube-api-access-2n7mj\") pod \"ovnkube-node-z889p\" (UID: \"1dda4441-e788-4fb1-8876-f736168d1c74\") " pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.575820 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.575795 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bncs\" (UniqueName: \"kubernetes.io/projected/9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816-kube-api-access-8bncs\") pod \"node-resolver-dwl4v\" (UID: \"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816\") " pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.577239 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.577210 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqknb\" (UniqueName: \"kubernetes.io/projected/52e6d6c8-f788-4a30-b59a-81fc53754d7b-kube-api-access-wqknb\") pod \"iptables-alerter-9lgj5\" (UID: \"52e6d6c8-f788-4a30-b59a-81fc53754d7b\") " pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.578116 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.578093 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xhps\" (UniqueName: \"kubernetes.io/projected/96c383c4-32ae-4252-bda3-6575b8d24b76-kube-api-access-7xhps\") pod \"node-ca-js8z5\" (UID: \"96c383c4-32ae-4252-bda3-6575b8d24b76\") " pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.643078 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.643035 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jbg76" Jul 21 16:24:49.652405 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.652374 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" Jul 21 16:24:49.661046 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.661024 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-clmbb" Jul 21 16:24:49.664047 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.664021 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34fa482_21e3_448c_ab10_2ca59ef86ae1.slice/crio-f3d5eca2dfbf689d9e449c9d7a6249165d620b802232aa31db40b4167b50ee77 WatchSource:0}: Error finding container f3d5eca2dfbf689d9e449c9d7a6249165d620b802232aa31db40b4167b50ee77: Status 404 returned error can't find the container with id f3d5eca2dfbf689d9e449c9d7a6249165d620b802232aa31db40b4167b50ee77 Jul 21 16:24:49.667637 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.667614 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:24:49.675988 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.675908 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5z485" Jul 21 16:24:49.676312 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.676288 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab204405_ee64_4517_9261_427c0a773d69.slice/crio-52347fbec3609a506dfc1b40cbf6794e496463dcb7270fa9a3abef59e8610cd4 WatchSource:0}: Error finding container 52347fbec3609a506dfc1b40cbf6794e496463dcb7270fa9a3abef59e8610cd4: Status 404 returned error can't find the container with id 52347fbec3609a506dfc1b40cbf6794e496463dcb7270fa9a3abef59e8610cd4 Jul 21 16:24:49.684222 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.684197 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5791f270_f605_4255_8fad_d441db9d0be3.slice/crio-32e073b48081c58d216285a9c3a29eb2a2621c4e65d28d3e914069456b5168e7 WatchSource:0}: Error finding container 32e073b48081c58d216285a9c3a29eb2a2621c4e65d28d3e914069456b5168e7: Status 404 returned error can't find the container with id 32e073b48081c58d216285a9c3a29eb2a2621c4e65d28d3e914069456b5168e7 Jul 21 16:24:49.687662 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.687469 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:24:49.693165 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.691613 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda59b6872_6c27_49c0_83b1_c6e3ecb3ed95.slice/crio-5e1a978cf6c65aed23520ba81df6a5d9358d32847555a49ba72a0afb456513b4 WatchSource:0}: Error finding container 5e1a978cf6c65aed23520ba81df6a5d9358d32847555a49ba72a0afb456513b4: Status 404 returned error can't find the container with id 5e1a978cf6c65aed23520ba81df6a5d9358d32847555a49ba72a0afb456513b4 Jul 21 16:24:49.694751 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.694590 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9lgj5" Jul 21 16:24:49.699603 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.699570 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dda4441_e788_4fb1_8876_f736168d1c74.slice/crio-0f05a2aa4c7fbafb2f00e24370f31f10306ba16ddca4f4b2e7b231e6aadb5687 WatchSource:0}: Error finding container 0f05a2aa4c7fbafb2f00e24370f31f10306ba16ddca4f4b2e7b231e6aadb5687: Status 404 returned error can't find the container with id 0f05a2aa4c7fbafb2f00e24370f31f10306ba16ddca4f4b2e7b231e6aadb5687 Jul 21 16:24:49.704182 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.704134 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dwl4v" Jul 21 16:24:49.705994 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.705890 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e6d6c8_f788_4a30_b59a_81fc53754d7b.slice/crio-c5c9f931f44ce480826b8ea10ec4d887478ab0053ac2299e9f308da4535cb076 WatchSource:0}: Error finding container c5c9f931f44ce480826b8ea10ec4d887478ab0053ac2299e9f308da4535cb076: Status 404 returned error can't find the container with id c5c9f931f44ce480826b8ea10ec4d887478ab0053ac2299e9f308da4535cb076 Jul 21 16:24:49.715976 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.715364 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d59dd5a_8f0a_4ad5_b3e2_d1e89eff7816.slice/crio-2821912ce3d439d6fa90f57cf8988be50368ef2d0b596885e657c83d396f8e94 WatchSource:0}: Error finding container 2821912ce3d439d6fa90f57cf8988be50368ef2d0b596885e657c83d396f8e94: Status 404 returned error can't find the container with id 2821912ce3d439d6fa90f57cf8988be50368ef2d0b596885e657c83d396f8e94 Jul 21 16:24:49.716228 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.716137 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-js8z5" Jul 21 16:24:49.728817 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:24:49.728789 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c383c4_32ae_4252_bda3_6575b8d24b76.slice/crio-bdcdecccff8b0cda72fcd99cddad1450693f5b6ff7845edd6d5b66578612ad7a WatchSource:0}: Error finding container bdcdecccff8b0cda72fcd99cddad1450693f5b6ff7845edd6d5b66578612ad7a: Status 404 returned error can't find the container with id bdcdecccff8b0cda72fcd99cddad1450693f5b6ff7845edd6d5b66578612ad7a Jul 21 16:24:49.756553 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:49.756523 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 16:24:50.062278 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.062180 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:50.062278 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.062229 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:50.062521 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.062374 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:50.062521 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.062399 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:50.062521 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.062461 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret podName:44dab062-86b8-4849-9d19-75299742a580 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:51.062439151 +0000 UTC m=+4.101274796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret") pod "global-pull-secret-syncer-9t8kx" (UID: "44dab062-86b8-4849-9d19-75299742a580") : object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:50.062521 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.062483 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs podName:64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd nodeName:}" failed. No retries permitted until 2026-07-21 16:24:51.062472764 +0000 UTC m=+4.101308407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs") pod "network-metrics-daemon-bglxt" (UID: "64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:50.163167 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.163133 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:50.163330 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.163276 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 16:24:50.163330 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.163304 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 16:24:50.163330 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.163316 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5d5kz for pod openshift-network-diagnostics/network-check-target-6r7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:50.163486 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.163380 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz podName:2af8f398-2b56-4964-a136-86ff864cbb35 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:51.163360612 +0000 UTC m=+4.202196275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5d5kz" (UniqueName: "kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz") pod "network-check-target-6r7qk" (UID: "2af8f398-2b56-4964-a136-86ff864cbb35") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:50.388107 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.387990 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 16:19:49 +0000 UTC" deadline="2028-03-19 21:43:47.747374953 +0000 UTC" Jul 21 16:24:50.388107 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.388043 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14573h18m57.359335162s" Jul 21 16:24:50.482492 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.482463 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:50.482676 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:50.482591 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:24:50.503215 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.503182 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dwl4v" event={"ID":"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816","Type":"ContainerStarted","Data":"2821912ce3d439d6fa90f57cf8988be50368ef2d0b596885e657c83d396f8e94"} Jul 21 16:24:50.504349 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.504325 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9lgj5" event={"ID":"52e6d6c8-f788-4a30-b59a-81fc53754d7b","Type":"ContainerStarted","Data":"c5c9f931f44ce480826b8ea10ec4d887478ab0053ac2299e9f308da4535cb076"} Jul 21 16:24:50.505423 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.505396 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xdwg8" event={"ID":"5791f270-f605-4255-8fad-d441db9d0be3","Type":"ContainerStarted","Data":"32e073b48081c58d216285a9c3a29eb2a2621c4e65d28d3e914069456b5168e7"} Jul 21 16:24:50.506621 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.506601 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-clmbb" event={"ID":"ab204405-ee64-4517-9261-427c0a773d69","Type":"ContainerStarted","Data":"52347fbec3609a506dfc1b40cbf6794e496463dcb7270fa9a3abef59e8610cd4"} Jul 21 16:24:50.507859 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.507818 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" event={"ID":"a34fa482-21e3-448c-ab10-2ca59ef86ae1","Type":"ContainerStarted","Data":"f3d5eca2dfbf689d9e449c9d7a6249165d620b802232aa31db40b4167b50ee77"} Jul 21 16:24:50.508992 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.508956 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-js8z5" event={"ID":"96c383c4-32ae-4252-bda3-6575b8d24b76","Type":"ContainerStarted","Data":"bdcdecccff8b0cda72fcd99cddad1450693f5b6ff7845edd6d5b66578612ad7a"} Jul 21 16:24:50.510369 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.510344 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"0f05a2aa4c7fbafb2f00e24370f31f10306ba16ddca4f4b2e7b231e6aadb5687"} Jul 21 16:24:50.511430 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.511400 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerStarted","Data":"5e1a978cf6c65aed23520ba81df6a5d9358d32847555a49ba72a0afb456513b4"} Jul 21 16:24:50.512550 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:50.512530 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jbg76" event={"ID":"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be","Type":"ContainerStarted","Data":"861d67046c62bdcdeae49af499a112be5073ed2e2b2b016628d1bfd086f77fb8"} Jul 21 16:24:51.072270 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:51.072224 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:51.072463 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:51.072276 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:51.072463 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.072447 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:51.072586 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.072454 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:51.072586 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.072521 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs podName:64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd nodeName:}" failed. No retries permitted until 2026-07-21 16:24:53.072501069 +0000 UTC m=+6.111336714 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs") pod "network-metrics-daemon-bglxt" (UID: "64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:51.072586 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.072542 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret podName:44dab062-86b8-4849-9d19-75299742a580 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:53.072531755 +0000 UTC m=+6.111367402 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret") pod "global-pull-secret-syncer-9t8kx" (UID: "44dab062-86b8-4849-9d19-75299742a580") : object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:51.173370 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:51.173329 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:51.173580 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.173560 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 16:24:51.173652 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.173591 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 16:24:51.173652 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.173606 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5d5kz for pod openshift-network-diagnostics/network-check-target-6r7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:51.173766 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.173672 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz podName:2af8f398-2b56-4964-a136-86ff864cbb35 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:53.173652557 +0000 UTC m=+6.212488216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5d5kz" (UniqueName: "kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz") pod "network-check-target-6r7qk" (UID: "2af8f398-2b56-4964-a136-86ff864cbb35") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:51.388962 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:51.388838 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 16:19:49 +0000 UTC" deadline="2028-01-16 10:00:53.877856125 +0000 UTC" Jul 21 16:24:51.388962 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:51.388899 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13049h36m2.488961793s" Jul 21 16:24:51.484620 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:51.484020 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:51.484620 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:51.484037 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:51.484620 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.484219 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:24:51.484620 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:51.484396 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:24:52.483795 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:52.483467 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:52.484297 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:52.483857 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:24:52.530992 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:52.530245 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" event={"ID":"90ea7be93df65d66da652d9417af881e","Type":"ContainerStarted","Data":"97a679d475d0d4d7a799166d8debacade559e74c9a6163fdc3204dab0524f484"} Jul 21 16:24:52.542828 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:52.538155 2557 generic.go:356] "Generic (PLEG): container finished" podID="15fcf4b6dde762c7ce3ec0cc2a1506cd" containerID="7bec79fb8c1dc3319b9d1cc8bf3ea1d96a1c908544122f4d3bfd650633963b60" exitCode=0 Jul 21 16:24:52.542828 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:52.538205 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"15fcf4b6dde762c7ce3ec0cc2a1506cd","Type":"ContainerDied","Data":"7bec79fb8c1dc3319b9d1cc8bf3ea1d96a1c908544122f4d3bfd650633963b60"} Jul 21 16:24:52.576555 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:52.576487 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" podStartSLOduration=3.576463071 podStartE2EDuration="3.576463071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:24:52.554307738 +0000 UTC m=+5.593143407" watchObservedRunningTime="2026-07-21 16:24:52.576463071 +0000 UTC m=+5.615298738" Jul 21 16:24:53.091597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:53.091556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:53.091805 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:53.091612 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:53.091805 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.091737 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:53.091805 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.091805 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs podName:64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd nodeName:}" failed. No retries permitted until 2026-07-21 16:24:57.091785183 +0000 UTC m=+10.130620850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs") pod "network-metrics-daemon-bglxt" (UID: "64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:53.092253 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.092231 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:53.092339 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.092296 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret podName:44dab062-86b8-4849-9d19-75299742a580 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:57.092278802 +0000 UTC m=+10.131114445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret") pod "global-pull-secret-syncer-9t8kx" (UID: "44dab062-86b8-4849-9d19-75299742a580") : object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:53.192304 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:53.192262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:53.192558 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.192523 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 16:24:53.192646 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.192562 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 16:24:53.192646 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.192576 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5d5kz for pod openshift-network-diagnostics/network-check-target-6r7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:53.192646 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.192639 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz podName:2af8f398-2b56-4964-a136-86ff864cbb35 nodeName:}" failed. No retries permitted until 2026-07-21 16:24:57.192617977 +0000 UTC m=+10.231453619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5d5kz" (UniqueName: "kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz") pod "network-check-target-6r7qk" (UID: "2af8f398-2b56-4964-a136-86ff864cbb35") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:53.484087 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:53.483997 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:53.484583 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.484152 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:24:53.484820 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:53.484800 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:53.484969 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:53.484944 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:24:54.483061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:54.483024 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:54.483251 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:54.483172 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:24:55.483658 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:55.483624 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:55.484193 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:55.483631 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:55.484193 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:55.483779 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:24:55.484193 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:55.483874 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:24:56.482712 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:56.482617 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:56.482944 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:56.482749 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:24:57.127061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:57.127018 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:57.127061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:57.127067 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:57.127526 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.127212 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:57.127526 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.127273 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs podName:64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd nodeName:}" failed. No retries permitted until 2026-07-21 16:25:05.127252951 +0000 UTC m=+18.166088608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs") pod "network-metrics-daemon-bglxt" (UID: "64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:24:57.127677 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.127655 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:57.127794 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.127719 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret podName:44dab062-86b8-4849-9d19-75299742a580 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:05.127702405 +0000 UTC m=+18.166538045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret") pod "global-pull-secret-syncer-9t8kx" (UID: "44dab062-86b8-4849-9d19-75299742a580") : object "kube-system"/"original-pull-secret" not registered Jul 21 16:24:57.228189 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:57.228153 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:57.228359 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.228334 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 16:24:57.228426 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.228362 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 16:24:57.228426 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.228374 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5d5kz for pod openshift-network-diagnostics/network-check-target-6r7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:57.228531 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.228432 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz podName:2af8f398-2b56-4964-a136-86ff864cbb35 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:05.228412145 +0000 UTC m=+18.267247791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5d5kz" (UniqueName: "kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz") pod "network-check-target-6r7qk" (UID: "2af8f398-2b56-4964-a136-86ff864cbb35") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:24:57.484119 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:57.483854 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:57.484119 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.483971 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:24:57.484357 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:57.484139 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:57.484357 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:57.484255 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:24:58.482782 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:58.482751 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:24:58.483166 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:58.482895 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:24:59.483791 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:59.483146 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:24:59.483791 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:59.483285 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:24:59.483791 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:24:59.483646 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:24:59.483791 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:24:59.483745 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:00.483123 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:00.483086 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:00.483308 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:00.483193 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:01.483459 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:01.483427 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:01.484000 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:01.483551 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:01.484067 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:01.484034 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:01.484188 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:01.484148 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:02.483277 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:02.483244 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:02.483458 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:02.483364 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:03.482663 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:03.482626 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:03.483157 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:03.482771 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:03.483157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:03.482778 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:03.483157 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:03.482946 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:04.482547 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:04.482498 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:04.482765 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:04.482627 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:05.195564 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:05.195477 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:05.195564 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:05.195528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:05.195811 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.195653 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:25:05.195811 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.195654 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 16:25:05.195811 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.195717 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs podName:64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd nodeName:}" failed. No retries permitted until 2026-07-21 16:25:21.195697369 +0000 UTC m=+34.234533013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs") pod "network-metrics-daemon-bglxt" (UID: "64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:25:05.195811 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.195733 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret podName:44dab062-86b8-4849-9d19-75299742a580 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:21.195726102 +0000 UTC m=+34.234561746 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret") pod "global-pull-secret-syncer-9t8kx" (UID: "44dab062-86b8-4849-9d19-75299742a580") : object "kube-system"/"original-pull-secret" not registered Jul 21 16:25:05.296133 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:05.296081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:05.296299 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.296200 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 16:25:05.296299 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.296221 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 16:25:05.296299 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.296235 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5d5kz for pod openshift-network-diagnostics/network-check-target-6r7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:25:05.296299 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.296301 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz podName:2af8f398-2b56-4964-a136-86ff864cbb35 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:21.296281758 +0000 UTC m=+34.335117398 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5d5kz" (UniqueName: "kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz") pod "network-check-target-6r7qk" (UID: "2af8f398-2b56-4964-a136-86ff864cbb35") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:25:05.483323 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:05.483233 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:05.483729 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.483383 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:05.483729 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:05.483440 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:05.483729 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:05.483543 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:06.483274 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:06.483246 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:06.483444 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:06.483366 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:07.483692 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:07.483659 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:07.484143 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:07.483781 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:07.484143 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:07.483834 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:07.484143 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:07.483898 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:08.483365 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:08.483325 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:08.483542 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:08.483438 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:09.483241 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.483143 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:09.483618 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:09.483289 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:09.483618 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.483536 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:09.483712 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:09.483649 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:09.581396 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.581237 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"15fcf4b6dde762c7ce3ec0cc2a1506cd","Type":"ContainerStarted","Data":"fb879f1f62cb7f30363127e4f9ffe6ffbcdae237b1f10821c6899539e84f4568"} Jul 21 16:25:09.583482 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.583299 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerStarted","Data":"7f445ae3d43b258fe8ea9ad1b323194778a15aed197db6670a246244f28af276"} Jul 21 16:25:09.588095 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.588074 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dwl4v" event={"ID":"9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816","Type":"ContainerStarted","Data":"43d9254a5f788529a8f171890949c4be3a176393932c112199bdac7f1d35f740"} Jul 21 16:25:09.589601 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.589547 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xdwg8" event={"ID":"5791f270-f605-4255-8fad-d441db9d0be3","Type":"ContainerStarted","Data":"8c29838902af1bd6d1c695e250019be5b87b33a5a3282f30d19fd5659a6df94a"} Jul 21 16:25:09.591107 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.591077 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-clmbb" event={"ID":"ab204405-ee64-4517-9261-427c0a773d69","Type":"ContainerStarted","Data":"5526bd2041649699df419aaf3ab2049ea37e1659452fbf2f20bcea6940136b82"} Jul 21 16:25:09.592679 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.592630 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" event={"ID":"a34fa482-21e3-448c-ab10-2ca59ef86ae1","Type":"ContainerStarted","Data":"bbc3c6962032916e98a8780983a746ef48f54e118ae25c16118ed2ac52fa8199"} Jul 21 16:25:09.598906 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.598864 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" podStartSLOduration=20.598807606 podStartE2EDuration="20.598807606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:09.598080324 +0000 UTC m=+22.636915990" watchObservedRunningTime="2026-07-21 16:25:09.598807606 +0000 UTC m=+22.637643267" Jul 21 16:25:09.613031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.612988 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xdwg8" podStartSLOduration=6.634133508 podStartE2EDuration="21.612972623s" totalImagesPullingTime="14.978839115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:09.612783341 +0000 UTC m=+22.651619002" watchObservedRunningTime="2026-07-21 16:25:09.612972623 +0000 UTC m=+22.651808285" Jul 21 16:25:09.627288 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.627232 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dwl4v" podStartSLOduration=2.104967606 podStartE2EDuration="21.627212887s" totalImagesPullingTime="19.522245281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:09.627128797 +0000 UTC m=+22.665964462" watchObservedRunningTime="2026-07-21 16:25:09.627212887 +0000 UTC m=+22.666048550" Jul 21 16:25:09.645335 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:09.645291 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-clmbb" podStartSLOduration=2.082834893 podStartE2EDuration="21.645278874s" totalImagesPullingTime="19.562443981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:09.645231817 +0000 UTC m=+22.684067478" watchObservedRunningTime="2026-07-21 16:25:09.645278874 +0000 UTC m=+22.684114536" Jul 21 16:25:10.384513 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.384489 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:25:10.385342 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.385322 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:25:10.483455 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.483429 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:10.484098 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:10.483517 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:10.595641 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.595610 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-js8z5" event={"ID":"96c383c4-32ae-4252-bda3-6575b8d24b76","Type":"ContainerStarted","Data":"d51e9c6fc443900818d4eb476250425760691f557052506b377a1936c0cf7a57"} Jul 21 16:25:10.597035 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.597013 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"1d430c59330d1b1c6b1e5278970aac81c84cd270fbd4e84c78533a469ce428fe"} Jul 21 16:25:10.598266 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.598222 2557 generic.go:356] "Generic (PLEG): container finished" podID="a59b6872-6c27-49c0-83b1-c6e3ecb3ed95" containerID="7f445ae3d43b258fe8ea9ad1b323194778a15aed197db6670a246244f28af276" exitCode=0 Jul 21 16:25:10.598345 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.598298 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerDied","Data":"7f445ae3d43b258fe8ea9ad1b323194778a15aed197db6670a246244f28af276"} Jul 21 16:25:10.599813 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.599786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jbg76" event={"ID":"ed8ef0ec-917a-42b0-bcf8-2bd6aaee67be","Type":"ContainerStarted","Data":"53ac2b4d687c8a650c1385fc6e75026eb639ba513a21acf45a870237ed53070e"} Jul 21 16:25:10.601320 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.601298 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9lgj5" event={"ID":"52e6d6c8-f788-4a30-b59a-81fc53754d7b","Type":"ContainerStarted","Data":"a8d8f2945414d646263f2e13e4bca43bedb3064db9d266cb0ca6b38967b8eb00"} Jul 21 16:25:10.601969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.601898 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:25:10.602437 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.602422 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xdwg8" Jul 21 16:25:10.611660 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.611627 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-js8z5" podStartSLOduration=3.102128373 podStartE2EDuration="22.611617077s" totalImagesPullingTime="19.509488704s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:10.611346882 +0000 UTC m=+23.650182544" watchObservedRunningTime="2026-07-21 16:25:10.611617077 +0000 UTC m=+23.650452733" Jul 21 16:25:10.655087 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.654933 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jbg76" podStartSLOduration=3.032044033 podStartE2EDuration="22.6549161s" totalImagesPullingTime="19.622872067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:10.654601218 +0000 UTC m=+23.693436880" watchObservedRunningTime="2026-07-21 16:25:10.6549161 +0000 UTC m=+23.693751761" Jul 21 16:25:10.690606 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.690564 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9lgj5" podStartSLOduration=3.15921674 podStartE2EDuration="22.690546352s" totalImagesPullingTime="19.531329612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:10.690436571 +0000 UTC m=+23.729272232" watchObservedRunningTime="2026-07-21 16:25:10.690546352 +0000 UTC m=+23.729382013" Jul 21 16:25:10.734997 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:10.734807 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 16:25:11.421456 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.421336 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T16:25:10.73491638Z","UUID":"c3b74249-c22e-4a42-9735-56710abe5e13","Handler":null,"Name":"","Endpoint":""} Jul 21 16:25:11.423081 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.423056 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 16:25:11.423222 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.423102 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 16:25:11.483513 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.483483 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:11.484018 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:11.483592 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:11.484018 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.483635 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:11.484018 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:11.483739 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:11.606267 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.606231 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" event={"ID":"a34fa482-21e3-448c-ab10-2ca59ef86ae1","Type":"ContainerStarted","Data":"c80d3982875c3f14637ad4587ca6a0f36b09a0478b808a65677a8d4f791cdbea"} Jul 21 16:25:11.609908 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.609866 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"20c72241099a9eba4b915152409b0b4721df6f8f4200693e86552aa13cc40b18"} Jul 21 16:25:11.610058 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.609947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"129b23440eaa53a919126ad341547a19e193340a352220b70d8e3c43abae602d"} Jul 21 16:25:11.610058 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.609998 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"27520009fdea22505faaf60643f82a6ae58e3eea90d9250918131325172e240f"} Jul 21 16:25:11.610058 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.610043 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"55d59e0c570b79079d9f2580e9396efdf0455408a4b4e8303978fca82e951a73"} Jul 21 16:25:11.610238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:11.610088 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"3148c7a7ca2cce75e198822fd35ce2f7d476507fd16a63ff98b1fe10dfb5d176"} Jul 21 16:25:12.483348 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:12.483166 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:12.483508 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:12.483428 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:12.614641 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:12.614596 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" event={"ID":"a34fa482-21e3-448c-ab10-2ca59ef86ae1","Type":"ContainerStarted","Data":"01a5c3d0158c474c186274d0830c0bcd3a880cb5ebf10d62d8d16a49430e6713"} Jul 21 16:25:12.635745 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:12.635701 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pszhj" podStartSLOduration=2.731926575 podStartE2EDuration="24.635687525s" totalImagesPullingTime="21.90376095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:12.635658783 +0000 UTC m=+25.674494442" watchObservedRunningTime="2026-07-21 16:25:12.635687525 +0000 UTC m=+25.674523178" Jul 21 16:25:13.483488 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:13.483408 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:13.483639 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:13.483531 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:13.483639 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:13.483580 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:13.483756 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:13.483685 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:13.619655 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:13.619622 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"8cb5dd9be545e58ac6fd5e669cf01b3daeb75c2d840f7898cfa3adc10defa330"} Jul 21 16:25:14.482597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:14.482569 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:14.482761 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:14.482668 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:15.483150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:15.483059 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:15.483150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:15.483092 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:15.483659 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:15.483219 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:15.483659 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:15.483220 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:16.483156 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:16.482917 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:16.483508 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:16.483173 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:16.630048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:16.630018 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" event={"ID":"1dda4441-e788-4fb1-8876-f736168d1c74","Type":"ContainerStarted","Data":"b91bc9dcd8f65cd25620dd637658c9bdccaf3b23deffea7eadde4a00d74d8db2"} Jul 21 16:25:16.630272 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:16.630244 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:25:16.631699 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:16.631679 2557 generic.go:356] "Generic (PLEG): container finished" podID="a59b6872-6c27-49c0-83b1-c6e3ecb3ed95" containerID="58c2abcc675daa7de7f4591033d4d81dbf029c71b97390f112c3752a766a4166" exitCode=0 Jul 21 16:25:16.631799 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:16.631708 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerDied","Data":"58c2abcc675daa7de7f4591033d4d81dbf029c71b97390f112c3752a766a4166"} Jul 21 16:25:16.646374 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:16.646354 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:25:16.665777 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:16.665737 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" podStartSLOduration=8.001929242 podStartE2EDuration="28.665723094s" totalImagesPullingTime="20.663793852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:16.665557507 +0000 UTC m=+29.704393171" watchObservedRunningTime="2026-07-21 16:25:16.665723094 +0000 UTC m=+29.704558755" Jul 21 16:25:17.483474 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.483444 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:17.483950 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:17.483586 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:17.483950 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.483637 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:17.483950 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:17.483772 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:17.634837 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.634794 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:25:17.634967 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.634898 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:25:17.650414 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.650396 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:25:17.829774 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.829599 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9t8kx"] Jul 21 16:25:17.829908 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.829898 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:17.829998 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:17.829980 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:17.831621 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.831598 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6r7qk"] Jul 21 16:25:17.831724 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.831696 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:17.831781 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:17.831766 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:17.834233 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.834213 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bglxt"] Jul 21 16:25:17.834318 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:17.834298 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:17.834398 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:17.834379 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:18.639404 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:18.639370 2557 generic.go:356] "Generic (PLEG): container finished" podID="a59b6872-6c27-49c0-83b1-c6e3ecb3ed95" containerID="6add68b80b48f76bf5afb709b6f335013308dedf3ec5c2b242b100017f05c527" exitCode=0 Jul 21 16:25:18.639798 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:18.639467 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerDied","Data":"6add68b80b48f76bf5afb709b6f335013308dedf3ec5c2b242b100017f05c527"} Jul 21 16:25:19.482654 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:19.482627 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:19.482654 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:19.482661 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:19.482881 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:19.482733 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:19.482881 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:19.482802 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:19.482962 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:19.482802 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:19.482962 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:19.482919 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:20.646881 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:20.646839 2557 generic.go:356] "Generic (PLEG): container finished" podID="a59b6872-6c27-49c0-83b1-c6e3ecb3ed95" containerID="83ec39ea79d1e183390b84bb122bd300e59070696a22e350ed8161212e76b16f" exitCode=0 Jul 21 16:25:20.647519 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:20.646884 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerDied","Data":"83ec39ea79d1e183390b84bb122bd300e59070696a22e350ed8161212e76b16f"} Jul 21 16:25:21.214207 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:21.214167 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:21.214395 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:21.214213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:21.214395 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.214292 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 16:25:21.214395 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.214365 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret podName:44dab062-86b8-4849-9d19-75299742a580 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:53.214350393 +0000 UTC m=+66.253186034 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret") pod "global-pull-secret-syncer-9t8kx" (UID: "44dab062-86b8-4849-9d19-75299742a580") : object "kube-system"/"original-pull-secret" not registered Jul 21 16:25:21.214395 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.214305 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:25:21.214618 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.214438 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs podName:64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd nodeName:}" failed. No retries permitted until 2026-07-21 16:25:53.214423037 +0000 UTC m=+66.253258677 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs") pod "network-metrics-daemon-bglxt" (UID: "64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 16:25:21.315150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:21.315117 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:21.315322 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.315306 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 16:25:21.315385 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.315332 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 16:25:21.315385 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.315344 2557 projected.go:196] Error preparing data for projected volume kube-api-access-5d5kz for pod openshift-network-diagnostics/network-check-target-6r7qk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:25:21.315464 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.315399 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz podName:2af8f398-2b56-4964-a136-86ff864cbb35 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:53.315385348 +0000 UTC m=+66.354220992 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5d5kz" (UniqueName: "kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz") pod "network-check-target-6r7qk" (UID: "2af8f398-2b56-4964-a136-86ff864cbb35") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 16:25:21.482702 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:21.482627 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:21.482856 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:21.482628 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:21.482856 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.482763 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bglxt" podUID="64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd" Jul 21 16:25:21.482856 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:21.482816 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:21.483007 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.482910 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9t8kx" podUID="44dab062-86b8-4849-9d19-75299742a580" Jul 21 16:25:21.483069 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:21.482989 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6r7qk" podUID="2af8f398-2b56-4964-a136-86ff864cbb35" Jul 21 16:25:23.328434 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.328355 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeReady" Jul 21 16:25:23.328964 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.328515 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 16:25:23.402686 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.402651 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b4888cb99-bz4lx"] Jul 21 16:25:23.404662 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.404641 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.410400 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.410370 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 16:25:23.410400 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.410390 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 16:25:23.410600 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.410443 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 16:25:23.412341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.412159 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-74hb8\"" Jul 21 16:25:23.420284 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.417434 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 16:25:23.438550 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.438464 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b4888cb99-bz4lx"] Jul 21 16:25:23.441495 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.441464 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7vszk"] Jul 21 16:25:23.443348 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.443331 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:23.447701 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.447681 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cxndm"] Jul 21 16:25:23.449546 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.449527 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.449870 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.449848 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 16:25:23.450084 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.450067 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 16:25:23.450164 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.450106 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 16:25:23.450755 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.450401 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kw8jm\"" Jul 21 16:25:23.455385 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.455367 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 16:25:23.455466 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.455410 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 16:25:23.455727 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.455711 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4z2m2\"" Jul 21 16:25:23.482211 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.482179 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cxndm"] Jul 21 16:25:23.482956 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.482929 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:23.483082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.482929 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:23.483082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.483063 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:23.486816 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.486795 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 16:25:23.487352 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.487311 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 16:25:23.488590 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.488570 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crp4d\"" Jul 21 16:25:23.488694 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.488628 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7vszk"] Jul 21 16:25:23.489083 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.489062 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 16:25:23.490049 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.490032 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lrbdq\"" Jul 21 16:25:23.490509 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.490353 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 16:25:23.532860 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.532839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.532967 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.532868 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-certificates\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.532967 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.532887 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08e60a6c-86b6-4064-9005-2a6751b253bf-ca-trust-extracted\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.532967 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.532914 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-bound-sa-token\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.533066 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.532978 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d5q7\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-kube-api-access-5d5q7\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.533066 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.533039 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-trusted-ca\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.533125 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.533071 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-image-registry-private-configuration\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.533125 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.533093 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-installation-pull-secrets\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.633810 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.633725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08e60a6c-86b6-4064-9005-2a6751b253bf-ca-trust-extracted\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.633810 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.633764 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqcnm\" (UniqueName: \"kubernetes.io/projected/6669ba74-1154-4860-b7d8-8f3482d4255b-kube-api-access-sqcnm\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.633810 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.633791 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6669ba74-1154-4860-b7d8-8f3482d4255b-tmp-dir\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.634101 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.633861 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-bound-sa-token\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634101 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.633918 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf308e21-a887-4859-93c3-5dfb60a312ef-cert\") pod \"ingress-canary-7vszk\" (UID: \"cf308e21-a887-4859-93c3-5dfb60a312ef\") " pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:23.634101 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.633966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5q7\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-kube-api-access-5d5q7\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634101 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634005 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.634101 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-trusted-ca\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634101 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-image-registry-private-configuration\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634291 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634142 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-installation-pull-secrets\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634291 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634175 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08e60a6c-86b6-4064-9005-2a6751b253bf-ca-trust-extracted\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634291 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634176 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pmjc\" (UniqueName: \"kubernetes.io/projected/cf308e21-a887-4859-93c3-5dfb60a312ef-kube-api-access-9pmjc\") pod \"ingress-canary-7vszk\" (UID: \"cf308e21-a887-4859-93c3-5dfb60a312ef\") " pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:23.634433 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634318 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6669ba74-1154-4860-b7d8-8f3482d4255b-config-volume\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.634433 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634354 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634433 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634381 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-certificates\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634784 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.634757 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-certificates\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.634926 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:23.634788 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 16:25:23.634926 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:23.634807 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b4888cb99-bz4lx: secret "image-registry-tls" not found Jul 21 16:25:23.634926 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:23.634892 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls podName:08e60a6c-86b6-4064-9005-2a6751b253bf nodeName:}" failed. No retries permitted until 2026-07-21 16:25:24.134875145 +0000 UTC m=+37.173710801 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls") pod "image-registry-5b4888cb99-bz4lx" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf") : secret "image-registry-tls" not found Jul 21 16:25:23.635130 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.635111 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-trusted-ca\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.638362 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.638339 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-installation-pull-secrets\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.638459 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.638422 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-image-registry-private-configuration\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.676520 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.676487 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-bound-sa-token\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.677082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.677056 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d5q7\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-kube-api-access-5d5q7\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:23.735502 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.735472 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pmjc\" (UniqueName: \"kubernetes.io/projected/cf308e21-a887-4859-93c3-5dfb60a312ef-kube-api-access-9pmjc\") pod \"ingress-canary-7vszk\" (UID: \"cf308e21-a887-4859-93c3-5dfb60a312ef\") " pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:23.735758 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.735525 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6669ba74-1154-4860-b7d8-8f3482d4255b-config-volume\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.735758 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.735620 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqcnm\" (UniqueName: \"kubernetes.io/projected/6669ba74-1154-4860-b7d8-8f3482d4255b-kube-api-access-sqcnm\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.735758 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.735655 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6669ba74-1154-4860-b7d8-8f3482d4255b-tmp-dir\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.735758 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.735687 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf308e21-a887-4859-93c3-5dfb60a312ef-cert\") pod \"ingress-canary-7vszk\" (UID: \"cf308e21-a887-4859-93c3-5dfb60a312ef\") " pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:23.735758 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.735724 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.736191 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:23.735853 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 16:25:23.736191 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:23.735920 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls podName:6669ba74-1154-4860-b7d8-8f3482d4255b nodeName:}" failed. No retries permitted until 2026-07-21 16:25:24.23590185 +0000 UTC m=+37.274737510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls") pod "dns-default-cxndm" (UID: "6669ba74-1154-4860-b7d8-8f3482d4255b") : secret "dns-default-metrics-tls" not found Jul 21 16:25:23.736191 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.735996 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6669ba74-1154-4860-b7d8-8f3482d4255b-tmp-dir\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.736191 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.736126 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6669ba74-1154-4860-b7d8-8f3482d4255b-config-volume\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:23.738275 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.738254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf308e21-a887-4859-93c3-5dfb60a312ef-cert\") pod \"ingress-canary-7vszk\" (UID: \"cf308e21-a887-4859-93c3-5dfb60a312ef\") " pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:23.755035 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.755011 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pmjc\" (UniqueName: \"kubernetes.io/projected/cf308e21-a887-4859-93c3-5dfb60a312ef-kube-api-access-9pmjc\") pod \"ingress-canary-7vszk\" (UID: \"cf308e21-a887-4859-93c3-5dfb60a312ef\") " pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:23.755890 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:23.755865 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqcnm\" (UniqueName: \"kubernetes.io/projected/6669ba74-1154-4860-b7d8-8f3482d4255b-kube-api-access-sqcnm\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:24.053314 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:24.053239 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7vszk" Jul 21 16:25:24.138871 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:24.138835 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:24.139021 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:24.138931 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 16:25:24.139021 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:24.138957 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b4888cb99-bz4lx: secret "image-registry-tls" not found Jul 21 16:25:24.139021 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:24.139022 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls podName:08e60a6c-86b6-4064-9005-2a6751b253bf nodeName:}" failed. No retries permitted until 2026-07-21 16:25:25.139006758 +0000 UTC m=+38.177842397 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls") pod "image-registry-5b4888cb99-bz4lx" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf") : secret "image-registry-tls" not found Jul 21 16:25:24.239860 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:24.239802 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:24.240025 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:24.239989 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 16:25:24.240084 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:24.240078 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls podName:6669ba74-1154-4860-b7d8-8f3482d4255b nodeName:}" failed. No retries permitted until 2026-07-21 16:25:25.240056473 +0000 UTC m=+38.278892118 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls") pod "dns-default-cxndm" (UID: "6669ba74-1154-4860-b7d8-8f3482d4255b") : secret "dns-default-metrics-tls" not found Jul 21 16:25:25.147007 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:25.146963 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:25.147439 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:25.147099 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 16:25:25.147439 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:25.147115 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b4888cb99-bz4lx: secret "image-registry-tls" not found Jul 21 16:25:25.147439 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:25.147175 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls podName:08e60a6c-86b6-4064-9005-2a6751b253bf nodeName:}" failed. No retries permitted until 2026-07-21 16:25:27.147157189 +0000 UTC m=+40.185992829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls") pod "image-registry-5b4888cb99-bz4lx" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf") : secret "image-registry-tls" not found Jul 21 16:25:25.248498 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:25.248128 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:25.248498 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:25.248304 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 16:25:25.248498 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:25.248459 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls podName:6669ba74-1154-4860-b7d8-8f3482d4255b nodeName:}" failed. No retries permitted until 2026-07-21 16:25:27.248444128 +0000 UTC m=+40.287279771 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls") pod "dns-default-cxndm" (UID: "6669ba74-1154-4860-b7d8-8f3482d4255b") : secret "dns-default-metrics-tls" not found Jul 21 16:25:25.409223 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:25.409194 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7vszk"] Jul 21 16:25:25.412877 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:25.412788 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf308e21_a887_4859_93c3_5dfb60a312ef.slice/crio-3ee26db44a84024820ed7df08e032d4a116fdd4362ed435546126c83bde38c47 WatchSource:0}: Error finding container 3ee26db44a84024820ed7df08e032d4a116fdd4362ed435546126c83bde38c47: Status 404 returned error can't find the container with id 3ee26db44a84024820ed7df08e032d4a116fdd4362ed435546126c83bde38c47 Jul 21 16:25:25.662745 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:25.662661 2557 generic.go:356] "Generic (PLEG): container finished" podID="a59b6872-6c27-49c0-83b1-c6e3ecb3ed95" containerID="fd27be1d967afbffaa0d80ab6c14be6fc75118c12b82d178c03a320a828a50f2" exitCode=0 Jul 21 16:25:25.662920 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:25.662773 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerDied","Data":"fd27be1d967afbffaa0d80ab6c14be6fc75118c12b82d178c03a320a828a50f2"} Jul 21 16:25:25.666629 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:25.666594 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7vszk" event={"ID":"cf308e21-a887-4859-93c3-5dfb60a312ef","Type":"ContainerStarted","Data":"3ee26db44a84024820ed7df08e032d4a116fdd4362ed435546126c83bde38c47"} Jul 21 16:25:26.672245 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:26.672208 2557 generic.go:356] "Generic (PLEG): container finished" podID="a59b6872-6c27-49c0-83b1-c6e3ecb3ed95" containerID="ed4377526d8c912799786e406c8019a4e4e842ec0a30e8ad0d74aa171d9faaa5" exitCode=0 Jul 21 16:25:26.672697 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:26.672260 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerDied","Data":"ed4377526d8c912799786e406c8019a4e4e842ec0a30e8ad0d74aa171d9faaa5"} Jul 21 16:25:27.163633 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.163393 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:27.163633 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.163547 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 16:25:27.163633 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.163619 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b4888cb99-bz4lx: secret "image-registry-tls" not found Jul 21 16:25:27.163846 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.163669 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls podName:08e60a6c-86b6-4064-9005-2a6751b253bf nodeName:}" failed. No retries permitted until 2026-07-21 16:25:31.16365541 +0000 UTC m=+44.202491050 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls") pod "image-registry-5b4888cb99-bz4lx" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf") : secret "image-registry-tls" not found Jul 21 16:25:27.174670 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.174635 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj"] Jul 21 16:25:27.196772 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.196742 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-h2cdv"] Jul 21 16:25:27.196936 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.196878 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.200615 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.200592 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 21 16:25:27.201443 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.201425 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.202407 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.202390 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4ldgd\"" Jul 21 16:25:27.202535 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.202516 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 21 16:25:27.202603 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.202516 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.220477 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.220451 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj"] Jul 21 16:25:27.220608 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.220491 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-h2cdv"] Jul 21 16:25:27.220608 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.220577 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.225291 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.225268 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 21 16:25:27.225291 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.225274 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 21 16:25:27.225766 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.225696 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.226294 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.226273 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.226388 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.226371 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-r7p2b\"" Jul 21 16:25:27.233799 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.233776 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 21 16:25:27.264441 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264411 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23d93917-4b64-4728-a183-f39e4530367a-service-ca-bundle\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.264584 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264513 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23d93917-4b64-4728-a183-f39e4530367a-tmp\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.264584 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.264697 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264601 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8qrk\" (UniqueName: \"kubernetes.io/projected/b369a7f2-8765-4858-9bb0-ab0f52838e21-kube-api-access-w8qrk\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.264697 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264626 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prnhv\" (UniqueName: \"kubernetes.io/projected/23d93917-4b64-4728-a183-f39e4530367a-kube-api-access-prnhv\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.264793 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264719 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:27.264793 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264763 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b369a7f2-8765-4858-9bb0-ab0f52838e21-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.264900 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/23d93917-4b64-4728-a183-f39e4530367a-snapshots\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.264900 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264857 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23d93917-4b64-4728-a183-f39e4530367a-serving-cert\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.264900 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.264887 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23d93917-4b64-4728-a183-f39e4530367a-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.265031 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.264931 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 16:25:27.265031 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.264993 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls podName:6669ba74-1154-4860-b7d8-8f3482d4255b nodeName:}" failed. No retries permitted until 2026-07-21 16:25:31.264972082 +0000 UTC m=+44.303807739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls") pod "dns-default-cxndm" (UID: "6669ba74-1154-4860-b7d8-8f3482d4255b") : secret "dns-default-metrics-tls" not found Jul 21 16:25:27.270231 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.270197 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8"] Jul 21 16:25:27.290544 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.290504 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-59rmv"] Jul 21 16:25:27.290715 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.290636 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.293940 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.293879 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 21 16:25:27.293940 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.293899 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-r8qpn\"" Jul 21 16:25:27.294301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.294198 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 21 16:25:27.294301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.294262 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.294464 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.294326 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.306479 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.306456 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8"] Jul 21 16:25:27.306595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.306498 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-59rmv"] Jul 21 16:25:27.306595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.306544 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.309367 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.309344 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.310447 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.310428 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 21 16:25:27.310547 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.310491 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cl4fd\"" Jul 21 16:25:27.310547 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.310496 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 21 16:25:27.310547 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.310433 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.317430 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.317412 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 21 16:25:27.365790 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23d93917-4b64-4728-a183-f39e4530367a-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.365969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365810 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38bb2b9c-3136-418d-a21d-1d2759954f76-config\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.365969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365841 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23d93917-4b64-4728-a183-f39e4530367a-tmp\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.365969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365876 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8qrk\" (UniqueName: \"kubernetes.io/projected/b369a7f2-8765-4858-9bb0-ab0f52838e21-kube-api-access-w8qrk\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.365969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365902 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc389b39-7c81-4b6a-aba7-920c43ad61b4-serving-cert\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.365969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365939 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23d93917-4b64-4728-a183-f39e4530367a-serving-cert\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.365969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365957 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38bb2b9c-3136-418d-a21d-1d2759954f76-serving-cert\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.365979 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv85c\" (UniqueName: \"kubernetes.io/projected/bc389b39-7c81-4b6a-aba7-920c43ad61b4-kube-api-access-nv85c\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366001 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23d93917-4b64-4728-a183-f39e4530367a-service-ca-bundle\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366050 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc389b39-7c81-4b6a-aba7-920c43ad61b4-trusted-ca\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b369a7f2-8765-4858-9bb0-ab0f52838e21-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366168 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prnhv\" (UniqueName: \"kubernetes.io/projected/23d93917-4b64-4728-a183-f39e4530367a-kube-api-access-prnhv\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.366178 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366206 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvg4\" (UniqueName: \"kubernetes.io/projected/38bb2b9c-3136-418d-a21d-1d2759954f76-kube-api-access-cfvg4\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.366259 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.366258 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls podName:b369a7f2-8765-4858-9bb0-ab0f52838e21 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:27.86623597 +0000 UTC m=+40.905071613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jxxfj" (UID: "b369a7f2-8765-4858-9bb0-ab0f52838e21") : secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:27.366754 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366272 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23d93917-4b64-4728-a183-f39e4530367a-tmp\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.366754 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366282 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc389b39-7c81-4b6a-aba7-920c43ad61b4-config\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.366754 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366332 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/23d93917-4b64-4728-a183-f39e4530367a-snapshots\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.366754 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366701 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23d93917-4b64-4728-a183-f39e4530367a-service-ca-bundle\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.366982 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366877 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23d93917-4b64-4728-a183-f39e4530367a-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.366982 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.366944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/23d93917-4b64-4728-a183-f39e4530367a-snapshots\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.367087 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.367068 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b369a7f2-8765-4858-9bb0-ab0f52838e21-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.371154 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.371131 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23d93917-4b64-4728-a183-f39e4530367a-serving-cert\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.377122 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.377059 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l"] Jul 21 16:25:27.379638 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.379614 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prnhv\" (UniqueName: \"kubernetes.io/projected/23d93917-4b64-4728-a183-f39e4530367a-kube-api-access-prnhv\") pod \"insights-operator-555d9d7885-h2cdv\" (UID: \"23d93917-4b64-4728-a183-f39e4530367a\") " pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.383798 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.383742 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8qrk\" (UniqueName: \"kubernetes.io/projected/b369a7f2-8765-4858-9bb0-ab0f52838e21-kube-api-access-w8qrk\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.395212 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.394895 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj"] Jul 21 16:25:27.395377 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.395290 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:27.398135 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.398113 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 21 16:25:27.398486 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.398149 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sppfl\"" Jul 21 16:25:27.398591 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.398509 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 21 16:25:27.410873 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.410846 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl"] Jul 21 16:25:27.411106 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.411086 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" Jul 21 16:25:27.417199 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.416942 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-scx2b\"" Jul 21 16:25:27.417199 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.417032 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.417199 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.417071 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.429327 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.429299 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt"] Jul 21 16:25:27.429446 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.429427 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.433238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.433207 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.433238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.433211 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 21 16:25:27.433394 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.433250 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.433394 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.433262 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 21 16:25:27.433394 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.433272 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-sbxnl\"" Jul 21 16:25:27.446924 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.446904 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-c99488556-jpl8w"] Jul 21 16:25:27.447020 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.446989 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" Jul 21 16:25:27.449944 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.449918 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-sch8p\"" Jul 21 16:25:27.467408 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467366 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e5edd2df-971a-4809-b26d-51110cac4ead-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.467408 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467404 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24sg8\" (UniqueName: \"kubernetes.io/projected/43b75aa3-b7f9-4aed-84bc-98e5f73d659a-kube-api-access-24sg8\") pod \"volume-data-source-validator-78cf4c6f6-ml2gj\" (UID: \"43b75aa3-b7f9-4aed-84bc-98e5f73d659a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" Jul 21 16:25:27.467602 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467439 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38bb2b9c-3136-418d-a21d-1d2759954f76-serving-cert\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.467602 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv85c\" (UniqueName: \"kubernetes.io/projected/bc389b39-7c81-4b6a-aba7-920c43ad61b4-kube-api-access-nv85c\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.467694 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467626 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc389b39-7c81-4b6a-aba7-920c43ad61b4-trusted-ca\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.467694 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467663 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e5edd2df-971a-4809-b26d-51110cac4ead-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.467792 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467692 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06057afc-c6f1-4735-a59b-20e7ada2f7c9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:27.467792 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467746 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvg4\" (UniqueName: \"kubernetes.io/projected/38bb2b9c-3136-418d-a21d-1d2759954f76-kube-api-access-cfvg4\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.467792 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djgr2\" (UniqueName: \"kubernetes.io/projected/e5edd2df-971a-4809-b26d-51110cac4ead-kube-api-access-djgr2\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.467977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc389b39-7c81-4b6a-aba7-920c43ad61b4-config\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.467977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467952 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2fgl\" (UniqueName: \"kubernetes.io/projected/5af73a0a-0f37-4f41-97d6-2537d6d93134-kube-api-access-q2fgl\") pod \"network-check-source-7bc5b6bf85-t66gt\" (UID: \"5af73a0a-0f37-4f41-97d6-2537d6d93134\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" Jul 21 16:25:27.468133 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.467982 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:27.468133 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.468040 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38bb2b9c-3136-418d-a21d-1d2759954f76-config\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.468133 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.468099 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc389b39-7c81-4b6a-aba7-920c43ad61b4-serving-cert\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.468838 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.468792 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bc389b39-7c81-4b6a-aba7-920c43ad61b4-trusted-ca\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.468970 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.468946 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bc389b39-7c81-4b6a-aba7-920c43ad61b4-config\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.469042 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.468980 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/38bb2b9c-3136-418d-a21d-1d2759954f76-config\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.469102 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.469050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l"] Jul 21 16:25:27.469102 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.469082 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj"] Jul 21 16:25:27.469200 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.469106 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl"] Jul 21 16:25:27.469200 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.469127 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt"] Jul 21 16:25:27.469200 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.469159 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c99488556-jpl8w"] Jul 21 16:25:27.469339 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.469254 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.470507 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.470484 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/38bb2b9c-3136-418d-a21d-1d2759954f76-serving-cert\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.471232 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.471212 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bc389b39-7c81-4b6a-aba7-920c43ad61b4-serving-cert\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.473145 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.473124 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 21 16:25:27.473229 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.473195 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 21 16:25:27.473364 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.473349 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.473434 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.473417 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.473497 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.473455 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 21 16:25:27.473551 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.473534 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 21 16:25:27.479367 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.479346 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql"] Jul 21 16:25:27.486627 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.486601 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvg4\" (UniqueName: \"kubernetes.io/projected/38bb2b9c-3136-418d-a21d-1d2759954f76-kube-api-access-cfvg4\") pod \"service-ca-operator-5bcbc94c78-fbqg8\" (UID: \"38bb2b9c-3136-418d-a21d-1d2759954f76\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.489262 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.489221 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jbwgt\"" Jul 21 16:25:27.489418 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.489399 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv85c\" (UniqueName: \"kubernetes.io/projected/bc389b39-7c81-4b6a-aba7-920c43ad61b4-kube-api-access-nv85c\") pod \"console-operator-565b7b569d-59rmv\" (UID: \"bc389b39-7c81-4b6a-aba7-920c43ad61b4\") " pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.509692 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.509669 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql"] Jul 21 16:25:27.509842 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.509776 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:27.512791 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.512759 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 21 16:25:27.513098 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.513074 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 21 16:25:27.514950 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.514929 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:25:27.519748 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.519730 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pfc7p\"" Jul 21 16:25:27.533906 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.533882 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-h2cdv" Jul 21 16:25:27.568841 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568711 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e5edd2df-971a-4809-b26d-51110cac4ead-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.568841 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06057afc-c6f1-4735-a59b-20e7ada2f7c9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:27.568841 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568788 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-stats-auth\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.568841 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568815 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtgdf\" (UniqueName: \"kubernetes.io/projected/cc2b1af9-464a-4817-8788-b5078e54c4c6-kube-api-access-xtgdf\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568880 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djgr2\" (UniqueName: \"kubernetes.io/projected/e5edd2df-971a-4809-b26d-51110cac4ead-kube-api-access-djgr2\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568921 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxd78\" (UniqueName: \"kubernetes.io/projected/826477a9-45f6-4deb-ae6f-d758f49e3d75-kube-api-access-sxd78\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568954 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2fgl\" (UniqueName: \"kubernetes.io/projected/5af73a0a-0f37-4f41-97d6-2537d6d93134-kube-api-access-q2fgl\") pod \"network-check-source-7bc5b6bf85-t66gt\" (UID: \"5af73a0a-0f37-4f41-97d6-2537d6d93134\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.568984 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.569008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.569076 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.569101 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.569111 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 16:25:27.569150 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.569145 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-default-certificate\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.569641 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.569187 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert podName:06057afc-c6f1-4735-a59b-20e7ada2f7c9 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:28.069166279 +0000 UTC m=+41.108001925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4q94l" (UID: "06057afc-c6f1-4735-a59b-20e7ada2f7c9") : secret "networking-console-plugin-cert" not found Jul 21 16:25:27.569641 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.569212 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e5edd2df-971a-4809-b26d-51110cac4ead-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.569641 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.569290 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24sg8\" (UniqueName: \"kubernetes.io/projected/43b75aa3-b7f9-4aed-84bc-98e5f73d659a-kube-api-access-24sg8\") pod \"volume-data-source-validator-78cf4c6f6-ml2gj\" (UID: \"43b75aa3-b7f9-4aed-84bc-98e5f73d659a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" Jul 21 16:25:27.569800 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.569713 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e5edd2df-971a-4809-b26d-51110cac4ead-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.570052 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.570028 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06057afc-c6f1-4735-a59b-20e7ada2f7c9-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:27.571357 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.571337 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e5edd2df-971a-4809-b26d-51110cac4ead-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.579809 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.579782 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24sg8\" (UniqueName: \"kubernetes.io/projected/43b75aa3-b7f9-4aed-84bc-98e5f73d659a-kube-api-access-24sg8\") pod \"volume-data-source-validator-78cf4c6f6-ml2gj\" (UID: \"43b75aa3-b7f9-4aed-84bc-98e5f73d659a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" Jul 21 16:25:27.579982 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.579960 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2fgl\" (UniqueName: \"kubernetes.io/projected/5af73a0a-0f37-4f41-97d6-2537d6d93134-kube-api-access-q2fgl\") pod \"network-check-source-7bc5b6bf85-t66gt\" (UID: \"5af73a0a-0f37-4f41-97d6-2537d6d93134\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" Jul 21 16:25:27.580381 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.580359 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djgr2\" (UniqueName: \"kubernetes.io/projected/e5edd2df-971a-4809-b26d-51110cac4ead-kube-api-access-djgr2\") pod \"kube-storage-version-migrator-operator-7658d9b749-qknkl\" (UID: \"e5edd2df-971a-4809-b26d-51110cac4ead\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.602312 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.602254 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" Jul 21 16:25:27.616458 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.616428 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:27.669920 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.669815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:27.669920 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.669880 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.670125 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.669927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-default-certificate\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.670125 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.669988 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 16:25:27.670125 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.670002 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-stats-auth\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.670125 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.670029 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtgdf\" (UniqueName: \"kubernetes.io/projected/cc2b1af9-464a-4817-8788-b5078e54c4c6-kube-api-access-xtgdf\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.670125 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.670006 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 16:25:27.670125 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.670065 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls podName:826477a9-45f6-4deb-ae6f-d758f49e3d75 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:28.170045108 +0000 UTC m=+41.208880753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-29tql" (UID: "826477a9-45f6-4deb-ae6f-d758f49e3d75") : secret "samples-operator-tls" not found Jul 21 16:25:27.670367 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.670166 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxd78\" (UniqueName: \"kubernetes.io/projected/826477a9-45f6-4deb-ae6f-d758f49e3d75-kube-api-access-sxd78\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:27.670367 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.670214 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:28.170194393 +0000 UTC m=+41.209030037 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : secret "router-metrics-certs-default" not found Jul 21 16:25:27.670367 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.670257 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.670501 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.670450 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:28.170433892 +0000 UTC m=+41.209269533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : configmap references non-existent config key: service-ca.crt Jul 21 16:25:27.673033 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.672982 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-default-certificate\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.673033 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.673010 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-stats-auth\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.678261 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.678220 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5z485" event={"ID":"a59b6872-6c27-49c0-83b1-c6e3ecb3ed95","Type":"ContainerStarted","Data":"e156c9cc3ef0bcb8a74d6a4f58660e32ac4a7dc8a11b153c412ecc168c805834"} Jul 21 16:25:27.681586 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.681547 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxd78\" (UniqueName: \"kubernetes.io/projected/826477a9-45f6-4deb-ae6f-d758f49e3d75-kube-api-access-sxd78\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:27.681758 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.681740 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtgdf\" (UniqueName: \"kubernetes.io/projected/cc2b1af9-464a-4817-8788-b5078e54c4c6-kube-api-access-xtgdf\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:27.708740 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.708691 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5z485" podStartSLOduration=7.8495894889999995 podStartE2EDuration="39.70867742s" totalImagesPullingTime="31.859087931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:27.708502354 +0000 UTC m=+40.747338029" watchObservedRunningTime="2026-07-21 16:25:27.70867742 +0000 UTC m=+40.747513082" Jul 21 16:25:27.727622 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.727586 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" Jul 21 16:25:27.739743 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.739707 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" Jul 21 16:25:27.756706 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.756683 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" Jul 21 16:25:27.872835 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:27.872775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:27.873024 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.872937 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:27.873024 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:27.873010 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls podName:b369a7f2-8765-4858-9bb0-ab0f52838e21 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:28.872994645 +0000 UTC m=+41.911830285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jxxfj" (UID: "b369a7f2-8765-4858-9bb0-ab0f52838e21") : secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:28.076006 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.075782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:28.076353 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.076019 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 16:25:28.076353 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.076084 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert podName:06057afc-c6f1-4735-a59b-20e7ada2f7c9 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:29.076065894 +0000 UTC m=+42.114901551 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4q94l" (UID: "06057afc-c6f1-4735-a59b-20e7ada2f7c9") : secret "networking-console-plugin-cert" not found Jul 21 16:25:28.167011 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.166974 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-h2cdv"] Jul 21 16:25:28.176618 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.176593 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:28.176721 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.176629 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:28.176771 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.176734 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 16:25:28.176817 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.176779 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:28.176817 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.176790 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 16:25:28.176929 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.176796 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls podName:826477a9-45f6-4deb-ae6f-d758f49e3d75 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:29.176781353 +0000 UTC m=+42.215616992 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-29tql" (UID: "826477a9-45f6-4deb-ae6f-d758f49e3d75") : secret "samples-operator-tls" not found Jul 21 16:25:28.176929 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.176869 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:29.176854353 +0000 UTC m=+42.215690000 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : secret "router-metrics-certs-default" not found Jul 21 16:25:28.176929 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.176883 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:29.176877457 +0000 UTC m=+42.215713096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : configmap references non-existent config key: service-ca.crt Jul 21 16:25:28.178713 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.178692 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt"] Jul 21 16:25:28.337472 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:28.337402 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d93917_4b64_4728_a183_f39e4530367a.slice/crio-3294dd884a4fca7a7cff6ce4988924d755496c679966de4a68b612dcd2914d1c WatchSource:0}: Error finding container 3294dd884a4fca7a7cff6ce4988924d755496c679966de4a68b612dcd2914d1c: Status 404 returned error can't find the container with id 3294dd884a4fca7a7cff6ce4988924d755496c679966de4a68b612dcd2914d1c Jul 21 16:25:28.338317 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:28.338301 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af73a0a_0f37_4f41_97d6_2537d6d93134.slice/crio-5cd85c15f27862f6dbc73c40a4bdf173acdbc826c10ee7ef602c451562fb361d WatchSource:0}: Error finding container 5cd85c15f27862f6dbc73c40a4bdf173acdbc826c10ee7ef602c451562fb361d: Status 404 returned error can't find the container with id 5cd85c15f27862f6dbc73c40a4bdf173acdbc826c10ee7ef602c451562fb361d Jul 21 16:25:28.395428 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.395405 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj"] Jul 21 16:25:28.397998 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:28.397963 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b75aa3_b7f9_4aed_84bc_98e5f73d659a.slice/crio-a5ec0456a80928c51f69b9504f20ee084ec4fb82ed5dec955cc970eb5c035255 WatchSource:0}: Error finding container a5ec0456a80928c51f69b9504f20ee084ec4fb82ed5dec955cc970eb5c035255: Status 404 returned error can't find the container with id a5ec0456a80928c51f69b9504f20ee084ec4fb82ed5dec955cc970eb5c035255 Jul 21 16:25:28.399421 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.399337 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8"] Jul 21 16:25:28.400364 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.400346 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl"] Jul 21 16:25:28.401465 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:28.401363 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc389b39_7c81_4b6a_aba7_920c43ad61b4.slice/crio-c51c740e4780f761fba99519c497f206adff4890c7d482f9330a990fe535fffe WatchSource:0}: Error finding container c51c740e4780f761fba99519c497f206adff4890c7d482f9330a990fe535fffe: Status 404 returned error can't find the container with id c51c740e4780f761fba99519c497f206adff4890c7d482f9330a990fe535fffe Jul 21 16:25:28.401658 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.401636 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-59rmv"] Jul 21 16:25:28.401998 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:28.401974 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5edd2df_971a_4809_b26d_51110cac4ead.slice/crio-498b93d142bc24de14e267116843bdc4588cfcc4fd09e7c5319c0c15f213718d WatchSource:0}: Error finding container 498b93d142bc24de14e267116843bdc4588cfcc4fd09e7c5319c0c15f213718d: Status 404 returned error can't find the container with id 498b93d142bc24de14e267116843bdc4588cfcc4fd09e7c5319c0c15f213718d Jul 21 16:25:28.402532 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:28.402511 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38bb2b9c_3136_418d_a21d_1d2759954f76.slice/crio-fdf3a75092055390581e99a21ab90709d3f7abe6cd20eda9236de7355ee53e22 WatchSource:0}: Error finding container fdf3a75092055390581e99a21ab90709d3f7abe6cd20eda9236de7355ee53e22: Status 404 returned error can't find the container with id fdf3a75092055390581e99a21ab90709d3f7abe6cd20eda9236de7355ee53e22 Jul 21 16:25:28.681874 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.681765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" event={"ID":"bc389b39-7c81-4b6a-aba7-920c43ad61b4","Type":"ContainerStarted","Data":"c51c740e4780f761fba99519c497f206adff4890c7d482f9330a990fe535fffe"} Jul 21 16:25:28.682939 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.682915 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-h2cdv" event={"ID":"23d93917-4b64-4728-a183-f39e4530367a","Type":"ContainerStarted","Data":"3294dd884a4fca7a7cff6ce4988924d755496c679966de4a68b612dcd2914d1c"} Jul 21 16:25:28.683934 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.683914 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" event={"ID":"5af73a0a-0f37-4f41-97d6-2537d6d93134","Type":"ContainerStarted","Data":"5cd85c15f27862f6dbc73c40a4bdf173acdbc826c10ee7ef602c451562fb361d"} Jul 21 16:25:28.685299 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.685274 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7vszk" event={"ID":"cf308e21-a887-4859-93c3-5dfb60a312ef","Type":"ContainerStarted","Data":"26439c5491d7d02b4428674d9756af9800e517bf4952ac045a4e50c1d7036a3e"} Jul 21 16:25:28.686333 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.686308 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" event={"ID":"e5edd2df-971a-4809-b26d-51110cac4ead","Type":"ContainerStarted","Data":"498b93d142bc24de14e267116843bdc4588cfcc4fd09e7c5319c0c15f213718d"} Jul 21 16:25:28.687315 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.687295 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" event={"ID":"38bb2b9c-3136-418d-a21d-1d2759954f76","Type":"ContainerStarted","Data":"fdf3a75092055390581e99a21ab90709d3f7abe6cd20eda9236de7355ee53e22"} Jul 21 16:25:28.688308 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.688290 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" event={"ID":"43b75aa3-b7f9-4aed-84bc-98e5f73d659a","Type":"ContainerStarted","Data":"a5ec0456a80928c51f69b9504f20ee084ec4fb82ed5dec955cc970eb5c035255"} Jul 21 16:25:28.703301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.703257 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7vszk" podStartSLOduration=2.741621206 podStartE2EDuration="5.703244497s" totalImagesPullingTime="2.961623291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:28.702509972 +0000 UTC m=+41.741345634" watchObservedRunningTime="2026-07-21 16:25:28.703244497 +0000 UTC m=+41.742080159" Jul 21 16:25:28.884284 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:28.884246 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:28.884458 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.884418 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:28.884510 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:28.884495 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls podName:b369a7f2-8765-4858-9bb0-ab0f52838e21 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:30.884477416 +0000 UTC m=+43.923313075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jxxfj" (UID: "b369a7f2-8765-4858-9bb0-ab0f52838e21") : secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:29.085638 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:29.085602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:29.085927 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:29.085900 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 16:25:29.086025 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:29.085974 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert podName:06057afc-c6f1-4735-a59b-20e7ada2f7c9 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:31.085955405 +0000 UTC m=+44.124791050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4q94l" (UID: "06057afc-c6f1-4735-a59b-20e7ada2f7c9") : secret "networking-console-plugin-cert" not found Jul 21 16:25:29.186874 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:29.186815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:29.187068 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:29.186889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:29.187068 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:29.186978 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 16:25:29.187068 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:29.187052 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:29.187230 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:29.187199 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:31.187180927 +0000 UTC m=+44.226016571 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : configmap references non-existent config key: service-ca.crt Jul 21 16:25:29.187291 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:29.187274 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 16:25:29.187341 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:29.187307 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:31.187296388 +0000 UTC m=+44.226132033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : secret "router-metrics-certs-default" not found Jul 21 16:25:29.187394 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:29.187353 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls podName:826477a9-45f6-4deb-ae6f-d758f49e3d75 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:31.187330927 +0000 UTC m=+44.226166572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-29tql" (UID: "826477a9-45f6-4deb-ae6f-d758f49e3d75") : secret "samples-operator-tls" not found Jul 21 16:25:30.905598 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:30.905553 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:30.906117 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:30.905731 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:30.906117 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:30.905811 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls podName:b369a7f2-8765-4858-9bb0-ab0f52838e21 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:34.905789481 +0000 UTC m=+47.944625143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jxxfj" (UID: "b369a7f2-8765-4858-9bb0-ab0f52838e21") : secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:31.107382 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:31.107345 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:31.107538 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.107485 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 16:25:31.107578 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.107561 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert podName:06057afc-c6f1-4735-a59b-20e7ada2f7c9 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:35.107536555 +0000 UTC m=+48.146372195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4q94l" (UID: "06057afc-c6f1-4735-a59b-20e7ada2f7c9") : secret "networking-console-plugin-cert" not found Jul 21 16:25:31.208657 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:31.208622 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:31.208767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:31.208684 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:31.208767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:31.208716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:31.208767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:31.208736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:31.208891 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208799 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 16:25:31.208891 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208834 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b4888cb99-bz4lx: secret "image-registry-tls" not found Jul 21 16:25:31.208891 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208847 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 16:25:31.208891 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208848 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 16:25:31.208891 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208892 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls podName:08e60a6c-86b6-4064-9005-2a6751b253bf nodeName:}" failed. No retries permitted until 2026-07-21 16:25:39.208872484 +0000 UTC m=+52.247708125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls") pod "image-registry-5b4888cb99-bz4lx" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf") : secret "image-registry-tls" not found Jul 21 16:25:31.209034 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208908 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:35.20890121 +0000 UTC m=+48.247736849 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : secret "router-metrics-certs-default" not found Jul 21 16:25:31.209034 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208918 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls podName:826477a9-45f6-4deb-ae6f-d758f49e3d75 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:35.208912766 +0000 UTC m=+48.247748405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-29tql" (UID: "826477a9-45f6-4deb-ae6f-d758f49e3d75") : secret "samples-operator-tls" not found Jul 21 16:25:31.209034 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.208934 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:35.208927094 +0000 UTC m=+48.247762735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : configmap references non-existent config key: service-ca.crt Jul 21 16:25:31.310152 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:31.310113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:31.310323 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.310287 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 16:25:31.310380 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:31.310368 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls podName:6669ba74-1154-4860-b7d8-8f3482d4255b nodeName:}" failed. No retries permitted until 2026-07-21 16:25:39.31034652 +0000 UTC m=+52.349182177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls") pod "dns-default-cxndm" (UID: "6669ba74-1154-4860-b7d8-8f3482d4255b") : secret "dns-default-metrics-tls" not found Jul 21 16:25:34.945075 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:34.945048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:34.945386 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:34.945177 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:34.945386 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:34.945228 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls podName:b369a7f2-8765-4858-9bb0-ab0f52838e21 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:42.945211858 +0000 UTC m=+55.984047510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jxxfj" (UID: "b369a7f2-8765-4858-9bb0-ab0f52838e21") : secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:35.146693 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.146660 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:35.146851 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:35.146805 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 16:25:35.146902 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:35.146887 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert podName:06057afc-c6f1-4735-a59b-20e7ada2f7c9 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:43.146867782 +0000 UTC m=+56.185703428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4q94l" (UID: "06057afc-c6f1-4735-a59b-20e7ada2f7c9") : secret "networking-console-plugin-cert" not found Jul 21 16:25:35.248062 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.247948 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:35.248062 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.248005 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:35.248062 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.248025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:35.248298 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:35.248114 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 16:25:35.248298 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:35.248149 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:43.248123473 +0000 UTC m=+56.286959136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : configmap references non-existent config key: service-ca.crt Jul 21 16:25:35.248298 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:35.248176 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls podName:826477a9-45f6-4deb-ae6f-d758f49e3d75 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:43.248166132 +0000 UTC m=+56.287001774 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-29tql" (UID: "826477a9-45f6-4deb-ae6f-d758f49e3d75") : secret "samples-operator-tls" not found Jul 21 16:25:35.248298 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:35.248189 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 16:25:35.248298 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:35.248219 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:43.248206822 +0000 UTC m=+56.287042461 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : secret "router-metrics-certs-default" not found Jul 21 16:25:35.715842 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.715792 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" event={"ID":"5af73a0a-0f37-4f41-97d6-2537d6d93134","Type":"ContainerStarted","Data":"33386f6d51db97c5c529b4eadf00f2c2bdaf77471ce4a2927dd9af98c7ba0508"} Jul 21 16:25:35.717430 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.717401 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" event={"ID":"e5edd2df-971a-4809-b26d-51110cac4ead","Type":"ContainerStarted","Data":"bcc1615334bd94d0d6095003b4203a886bc24d8577c91657781f7fe18672730b"} Jul 21 16:25:35.719210 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.719166 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" event={"ID":"38bb2b9c-3136-418d-a21d-1d2759954f76","Type":"ContainerStarted","Data":"f2c2d2808bef0a3ec0ca5ef7c9dfdcaf8637fb6d855a33a151e77b93b027acdd"} Jul 21 16:25:35.720828 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.720800 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" event={"ID":"43b75aa3-b7f9-4aed-84bc-98e5f73d659a","Type":"ContainerStarted","Data":"afb24fcef6754a9e3121408af7823b72d711ac75b117d7f580f44a3bf82108ea"} Jul 21 16:25:35.723002 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.722888 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/0.log" Jul 21 16:25:35.723002 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.722926 2557 generic.go:356] "Generic (PLEG): container finished" podID="bc389b39-7c81-4b6a-aba7-920c43ad61b4" containerID="52e55429788abfd04251a48e5315bda855218f0ccd5eddbbe4ded8efc782c9bd" exitCode=255 Jul 21 16:25:35.723002 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.723001 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" event={"ID":"bc389b39-7c81-4b6a-aba7-920c43ad61b4","Type":"ContainerDied","Data":"52e55429788abfd04251a48e5315bda855218f0ccd5eddbbe4ded8efc782c9bd"} Jul 21 16:25:35.723219 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.723169 2557 scope.go:117] "RemoveContainer" containerID="52e55429788abfd04251a48e5315bda855218f0ccd5eddbbe4ded8efc782c9bd" Jul 21 16:25:35.725564 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.725146 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-h2cdv" event={"ID":"23d93917-4b64-4728-a183-f39e4530367a","Type":"ContainerStarted","Data":"28caa750c527cf26e924c546890a7a310929e3954cc10c70f43e463bc1b01c53"} Jul 21 16:25:35.733178 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.732633 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-t66gt" podStartSLOduration=2.158451081 podStartE2EDuration="8.732617008s" totalImagesPullingTime="6.574165927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:35.731326642 +0000 UTC m=+48.770162305" watchObservedRunningTime="2026-07-21 16:25:35.732617008 +0000 UTC m=+48.771452650" Jul 21 16:25:35.751155 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.751107 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-ml2gj" podStartSLOduration=2.307556702 podStartE2EDuration="8.751089336s" totalImagesPullingTime="6.443532634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:35.75066339 +0000 UTC m=+48.789499053" watchObservedRunningTime="2026-07-21 16:25:35.751089336 +0000 UTC m=+48.789924998" Jul 21 16:25:35.770850 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.770759 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" podStartSLOduration=2.243727512 podStartE2EDuration="8.770737658s" totalImagesPullingTime="6.527010146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:35.768439219 +0000 UTC m=+48.807274882" watchObservedRunningTime="2026-07-21 16:25:35.770737658 +0000 UTC m=+48.809573321" Jul 21 16:25:35.788368 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.787984 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-h2cdv" podStartSLOduration=2.22345916 podStartE2EDuration="8.787964413s" totalImagesPullingTime="6.564505253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:35.786971494 +0000 UTC m=+48.825807157" watchObservedRunningTime="2026-07-21 16:25:35.787964413 +0000 UTC m=+48.826800075" Jul 21 16:25:35.810770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:35.810714 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" podStartSLOduration=2.284264524 podStartE2EDuration="8.810679942s" totalImagesPullingTime="6.526415418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:35.808972755 +0000 UTC m=+48.847808410" watchObservedRunningTime="2026-07-21 16:25:35.810679942 +0000 UTC m=+48.849515605" Jul 21 16:25:36.494450 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.494415 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2frkk"] Jul 21 16:25:36.518037 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.518010 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2frkk"] Jul 21 16:25:36.518183 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.518126 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.520973 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.520953 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 16:25:36.521294 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.521275 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 16:25:36.522713 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.522576 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7w2v\"" Jul 21 16:25:36.561855 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.561806 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.562031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.561873 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.562031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.561972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/addf81be-facb-4540-8b4c-4960f292deb8-data-volume\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.562122 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.562056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/addf81be-facb-4540-8b4c-4960f292deb8-crio-socket\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.562122 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.562103 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjxp8\" (UniqueName: \"kubernetes.io/projected/addf81be-facb-4540-8b4c-4960f292deb8-kube-api-access-cjxp8\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.663237 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.663202 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.663412 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.663253 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.663412 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.663335 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/addf81be-facb-4540-8b4c-4960f292deb8-data-volume\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.663412 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:36.663320 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 16:25:36.663412 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:36.663363 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 16:25:36.663629 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:36.663428 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls podName:addf81be-facb-4540-8b4c-4960f292deb8 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:37.163407234 +0000 UTC m=+50.202242901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2frkk" (UID: "addf81be-facb-4540-8b4c-4960f292deb8") : secret "insights-runtime-extractor-tls" not found Jul 21 16:25:36.663629 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:36.663448 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm podName:addf81be-facb-4540-8b4c-4960f292deb8 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:37.16343865 +0000 UTC m=+50.202274300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2frkk" (UID: "addf81be-facb-4540-8b4c-4960f292deb8") : configmap "kube-rbac-proxy" not found Jul 21 16:25:36.663629 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.663506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/addf81be-facb-4540-8b4c-4960f292deb8-crio-socket\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.663629 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.663555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjxp8\" (UniqueName: \"kubernetes.io/projected/addf81be-facb-4540-8b4c-4960f292deb8-kube-api-access-cjxp8\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.663629 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.663588 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/addf81be-facb-4540-8b4c-4960f292deb8-crio-socket\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.663907 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.663683 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/addf81be-facb-4540-8b4c-4960f292deb8-data-volume\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.673449 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.673424 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjxp8\" (UniqueName: \"kubernetes.io/projected/addf81be-facb-4540-8b4c-4960f292deb8-kube-api-access-cjxp8\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:36.730631 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.730603 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:25:36.731101 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.731083 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/0.log" Jul 21 16:25:36.731246 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.731117 2557 generic.go:356] "Generic (PLEG): container finished" podID="bc389b39-7c81-4b6a-aba7-920c43ad61b4" containerID="4978f780bab1973abfd39907cb7a1373465bb962fd0565649e432d7ef16bdf09" exitCode=255 Jul 21 16:25:36.731246 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.731210 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" event={"ID":"bc389b39-7c81-4b6a-aba7-920c43ad61b4","Type":"ContainerDied","Data":"4978f780bab1973abfd39907cb7a1373465bb962fd0565649e432d7ef16bdf09"} Jul 21 16:25:36.731364 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.731274 2557 scope.go:117] "RemoveContainer" containerID="52e55429788abfd04251a48e5315bda855218f0ccd5eddbbe4ded8efc782c9bd" Jul 21 16:25:36.731599 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:36.731514 2557 scope.go:117] "RemoveContainer" containerID="4978f780bab1973abfd39907cb7a1373465bb962fd0565649e432d7ef16bdf09" Jul 21 16:25:36.731719 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:36.731683 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-59rmv_openshift-console-operator(bc389b39-7c81-4b6a-aba7-920c43ad61b4)\"" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" podUID="bc389b39-7c81-4b6a-aba7-920c43ad61b4" Jul 21 16:25:37.169538 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:37.169438 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:37.169710 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:37.169577 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 16:25:37.169710 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:37.169600 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:37.169710 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:37.169653 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm podName:addf81be-facb-4540-8b4c-4960f292deb8 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:38.169632367 +0000 UTC m=+51.208468006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2frkk" (UID: "addf81be-facb-4540-8b4c-4960f292deb8") : configmap "kube-rbac-proxy" not found Jul 21 16:25:37.169710 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:37.169692 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 16:25:37.169955 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:37.169745 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls podName:addf81be-facb-4540-8b4c-4960f292deb8 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:38.169734256 +0000 UTC m=+51.208569897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2frkk" (UID: "addf81be-facb-4540-8b4c-4960f292deb8") : secret "insights-runtime-extractor-tls" not found Jul 21 16:25:37.617548 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:37.617510 2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:37.617966 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:37.617591 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:37.735468 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:37.735441 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:25:37.735774 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:37.735758 2557 scope.go:117] "RemoveContainer" containerID="4978f780bab1973abfd39907cb7a1373465bb962fd0565649e432d7ef16bdf09" Jul 21 16:25:37.735972 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:37.735953 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-59rmv_openshift-console-operator(bc389b39-7c81-4b6a-aba7-920c43ad61b4)\"" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" podUID="bc389b39-7c81-4b6a-aba7-920c43ad61b4" Jul 21 16:25:38.088197 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:38.088167 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dwl4v_9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816/dns-node-resolver/0.log" Jul 21 16:25:38.179785 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:38.179753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:38.180001 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:38.179797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:38.180001 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:38.179926 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 16:25:38.180001 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:38.179991 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls podName:addf81be-facb-4540-8b4c-4960f292deb8 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:40.179976025 +0000 UTC m=+53.218811665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2frkk" (UID: "addf81be-facb-4540-8b4c-4960f292deb8") : secret "insights-runtime-extractor-tls" not found Jul 21 16:25:38.180362 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:38.180341 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/addf81be-facb-4540-8b4c-4960f292deb8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:38.738966 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:38.738938 2557 scope.go:117] "RemoveContainer" containerID="4978f780bab1973abfd39907cb7a1373465bb962fd0565649e432d7ef16bdf09" Jul 21 16:25:38.739338 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:38.739089 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-59rmv_openshift-console-operator(bc389b39-7c81-4b6a-aba7-920c43ad61b4)\"" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" podUID="bc389b39-7c81-4b6a-aba7-920c43ad61b4" Jul 21 16:25:39.098466 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:39.098393 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-js8z5_96c383c4-32ae-4252-bda3-6575b8d24b76/node-ca/0.log" Jul 21 16:25:39.289431 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:39.289390 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:39.289613 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:39.289546 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 16:25:39.289613 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:39.289564 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b4888cb99-bz4lx: secret "image-registry-tls" not found Jul 21 16:25:39.289685 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:39.289618 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls podName:08e60a6c-86b6-4064-9005-2a6751b253bf nodeName:}" failed. No retries permitted until 2026-07-21 16:25:55.289603096 +0000 UTC m=+68.328438735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls") pod "image-registry-5b4888cb99-bz4lx" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf") : secret "image-registry-tls" not found Jul 21 16:25:39.390157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:39.390068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:39.390309 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:39.390194 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 16:25:39.390309 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:39.390244 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls podName:6669ba74-1154-4860-b7d8-8f3482d4255b nodeName:}" failed. No retries permitted until 2026-07-21 16:25:55.390230757 +0000 UTC m=+68.429066397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls") pod "dns-default-cxndm" (UID: "6669ba74-1154-4860-b7d8-8f3482d4255b") : secret "dns-default-metrics-tls" not found Jul 21 16:25:39.705916 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:39.705842 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7vszk_cf308e21-a887-4859-93c3-5dfb60a312ef/serve-healthcheck-canary/0.log" Jul 21 16:25:40.196609 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:40.196569 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:40.197085 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:40.196759 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 16:25:40.197085 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:40.196866 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls podName:addf81be-facb-4540-8b4c-4960f292deb8 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:44.196843594 +0000 UTC m=+57.235679247 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2frkk" (UID: "addf81be-facb-4540-8b4c-4960f292deb8") : secret "insights-runtime-extractor-tls" not found Jul 21 16:25:43.024782 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:43.024740 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:43.025360 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.024934 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:43.025360 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.025037 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls podName:b369a7f2-8765-4858-9bb0-ab0f52838e21 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:59.025015261 +0000 UTC m=+72.063850916 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jxxfj" (UID: "b369a7f2-8765-4858-9bb0-ab0f52838e21") : secret "cluster-monitoring-operator-tls" not found Jul 21 16:25:43.225773 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:43.225734 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:43.225974 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.225905 2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 21 16:25:43.226016 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.225983 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert podName:06057afc-c6f1-4735-a59b-20e7ada2f7c9 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:59.225964535 +0000 UTC m=+72.264800178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4q94l" (UID: "06057afc-c6f1-4735-a59b-20e7ada2f7c9") : secret "networking-console-plugin-cert" not found Jul 21 16:25:43.326642 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:43.326547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:43.326642 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:43.326603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:43.326642 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:43.326623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:43.326915 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.326700 2557 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 21 16:25:43.326915 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.326705 2557 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 21 16:25:43.326915 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.326757 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls podName:826477a9-45f6-4deb-ae6f-d758f49e3d75 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:59.326744453 +0000 UTC m=+72.365580093 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-29tql" (UID: "826477a9-45f6-4deb-ae6f-d758f49e3d75") : secret "samples-operator-tls" not found Jul 21 16:25:43.326915 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.326780 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:59.326772251 +0000 UTC m=+72.365607890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : secret "router-metrics-certs-default" not found Jul 21 16:25:43.326915 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:43.326791 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle podName:cc2b1af9-464a-4817-8788-b5078e54c4c6 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:59.326785904 +0000 UTC m=+72.365621544 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle") pod "router-default-c99488556-jpl8w" (UID: "cc2b1af9-464a-4817-8788-b5078e54c4c6") : configmap references non-existent config key: service-ca.crt Jul 21 16:25:44.235989 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:44.235949 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:44.236504 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:44.236118 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 16:25:44.236504 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:25:44.236188 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls podName:addf81be-facb-4540-8b4c-4960f292deb8 nodeName:}" failed. No retries permitted until 2026-07-21 16:25:52.236171225 +0000 UTC m=+65.275006865 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2frkk" (UID: "addf81be-facb-4540-8b4c-4960f292deb8") : secret "insights-runtime-extractor-tls" not found Jul 21 16:25:49.652465 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:49.652437 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z889p" Jul 21 16:25:50.483090 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:50.483058 2557 scope.go:117] "RemoveContainer" containerID="4978f780bab1973abfd39907cb7a1373465bb962fd0565649e432d7ef16bdf09" Jul 21 16:25:50.777712 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:50.777634 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:25:50.778154 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:50.777709 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" event={"ID":"bc389b39-7c81-4b6a-aba7-920c43ad61b4","Type":"ContainerStarted","Data":"20494657190a176a82776afbe272f835c899fa45ce0d3c07f4897df65b6b8161"} Jul 21 16:25:50.778154 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:50.778082 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:50.794892 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:50.794839 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" podStartSLOduration=17.266806874 podStartE2EDuration="23.794808814s" totalImagesPullingTime="6.52800194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:50.794523953 +0000 UTC m=+63.833359616" watchObservedRunningTime="2026-07-21 16:25:50.794808814 +0000 UTC m=+63.833644475" Jul 21 16:25:51.127437 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:51.127333 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-59rmv" Jul 21 16:25:52.304832 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:52.304788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:52.307130 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:52.307098 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/addf81be-facb-4540-8b4c-4960f292deb8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2frkk\" (UID: \"addf81be-facb-4540-8b4c-4960f292deb8\") " pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:52.430655 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:52.430625 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-q7w2v\"" Jul 21 16:25:52.438311 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:52.438293 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2frkk" Jul 21 16:25:52.562725 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:52.562649 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2frkk"] Jul 21 16:25:52.565354 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:52.565326 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaddf81be_facb_4540_8b4c_4960f292deb8.slice/crio-a9012db5f4724fdd0b458bf60d9442baf1858be816d38f81847bc3b106412872 WatchSource:0}: Error finding container a9012db5f4724fdd0b458bf60d9442baf1858be816d38f81847bc3b106412872: Status 404 returned error can't find the container with id a9012db5f4724fdd0b458bf60d9442baf1858be816d38f81847bc3b106412872 Jul 21 16:25:52.785590 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:52.785550 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2frkk" event={"ID":"addf81be-facb-4540-8b4c-4960f292deb8","Type":"ContainerStarted","Data":"a9012db5f4724fdd0b458bf60d9442baf1858be816d38f81847bc3b106412872"} Jul 21 16:25:53.214609 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.214572 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:53.214973 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.214716 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:53.217974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.217941 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 16:25:53.218090 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.217942 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 16:25:53.234399 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.230992 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/44dab062-86b8-4849-9d19-75299742a580-original-pull-secret\") pod \"global-pull-secret-syncer-9t8kx\" (UID: \"44dab062-86b8-4849-9d19-75299742a580\") " pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:53.234399 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.231211 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd-metrics-certs\") pod \"network-metrics-daemon-bglxt\" (UID: \"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd\") " pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:53.315492 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.315451 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:53.318667 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.318615 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d5kz\" (UniqueName: \"kubernetes.io/projected/2af8f398-2b56-4964-a136-86ff864cbb35-kube-api-access-5d5kz\") pod \"network-check-target-6r7qk\" (UID: \"2af8f398-2b56-4964-a136-86ff864cbb35\") " pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:53.495958 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.495927 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9t8kx" Jul 21 16:25:53.507006 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.506984 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lrbdq\"" Jul 21 16:25:53.511990 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.511903 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-crp4d\"" Jul 21 16:25:53.514388 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.513949 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bglxt" Jul 21 16:25:53.519860 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.519810 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:53.676976 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.676887 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9t8kx"] Jul 21 16:25:53.680315 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:53.680280 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44dab062_86b8_4849_9d19_75299742a580.slice/crio-a75203f3823d1a3b404bd5c48a53091efd74b4d7ff565037eaaae04eb22a9ecb WatchSource:0}: Error finding container a75203f3823d1a3b404bd5c48a53091efd74b4d7ff565037eaaae04eb22a9ecb: Status 404 returned error can't find the container with id a75203f3823d1a3b404bd5c48a53091efd74b4d7ff565037eaaae04eb22a9ecb Jul 21 16:25:53.711896 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.711809 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bglxt"] Jul 21 16:25:53.714566 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:53.714536 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64fd93bf_b5a9_4a5e_a97d_29a7d9ea5fbd.slice/crio-31151af77a68f0508ae4aee261d2874e2470348477238934c7aa4c22807846a4 WatchSource:0}: Error finding container 31151af77a68f0508ae4aee261d2874e2470348477238934c7aa4c22807846a4: Status 404 returned error can't find the container with id 31151af77a68f0508ae4aee261d2874e2470348477238934c7aa4c22807846a4 Jul 21 16:25:53.729772 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.729747 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6r7qk"] Jul 21 16:25:53.732877 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:53.732850 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af8f398_2b56_4964_a136_86ff864cbb35.slice/crio-b3fba8a5a97170b3fd96bfa20398a6d6a185d23b0c6d5348cc0bc787d47b1006 WatchSource:0}: Error finding container b3fba8a5a97170b3fd96bfa20398a6d6a185d23b0c6d5348cc0bc787d47b1006: Status 404 returned error can't find the container with id b3fba8a5a97170b3fd96bfa20398a6d6a185d23b0c6d5348cc0bc787d47b1006 Jul 21 16:25:53.791014 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.790986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bglxt" event={"ID":"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd","Type":"ContainerStarted","Data":"31151af77a68f0508ae4aee261d2874e2470348477238934c7aa4c22807846a4"} Jul 21 16:25:53.792744 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.792708 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6r7qk" event={"ID":"2af8f398-2b56-4964-a136-86ff864cbb35","Type":"ContainerStarted","Data":"b3fba8a5a97170b3fd96bfa20398a6d6a185d23b0c6d5348cc0bc787d47b1006"} Jul 21 16:25:53.794128 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:53.794104 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9t8kx" event={"ID":"44dab062-86b8-4849-9d19-75299742a580","Type":"ContainerStarted","Data":"a75203f3823d1a3b404bd5c48a53091efd74b4d7ff565037eaaae04eb22a9ecb"} Jul 21 16:25:54.801851 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:54.801180 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6r7qk" event={"ID":"2af8f398-2b56-4964-a136-86ff864cbb35","Type":"ContainerStarted","Data":"c066e737833cf3a7b73ce2570a4d0bb699b19ba71a2e821df3409d668380b7b9"} Jul 21 16:25:54.802460 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:54.802386 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:25:54.820620 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:54.819923 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6r7qk" podStartSLOduration=66.819903187 podStartE2EDuration="1m6.819903187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:54.818614588 +0000 UTC m=+67.857450250" watchObservedRunningTime="2026-07-21 16:25:54.819903187 +0000 UTC m=+67.858738832" Jul 21 16:25:55.339717 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.339678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:55.343077 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.342696 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"image-registry-5b4888cb99-bz4lx\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:55.441214 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.441173 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:55.444131 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.444104 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6669ba74-1154-4860-b7d8-8f3482d4255b-metrics-tls\") pod \"dns-default-cxndm\" (UID: \"6669ba74-1154-4860-b7d8-8f3482d4255b\") " pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:55.519017 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.518987 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-74hb8\"" Jul 21 16:25:55.526674 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.526641 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:55.563718 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.563682 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4z2m2\"" Jul 21 16:25:55.571156 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.571125 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cxndm" Jul 21 16:25:55.807147 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:55.807111 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bglxt" event={"ID":"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd","Type":"ContainerStarted","Data":"c3c392cddd934c74ef429bb3855c7f54b66fc4f8790f4d175c9976469f8c62ee"} Jul 21 16:25:59.075099 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.075048 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:59.077877 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.077844 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b369a7f2-8765-4858-9bb0-ab0f52838e21-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jxxfj\" (UID: \"b369a7f2-8765-4858-9bb0-ab0f52838e21\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:59.276982 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.276797 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:59.280298 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.280273 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06057afc-c6f1-4735-a59b-20e7ada2f7c9-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4q94l\" (UID: \"06057afc-c6f1-4735-a59b-20e7ada2f7c9\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:59.310113 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.310077 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4ldgd\"" Jul 21 16:25:59.318619 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.317715 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" Jul 21 16:25:59.379472 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.379448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:59.379564 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.379503 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:59.379564 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.379533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:59.380705 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.380648 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc2b1af9-464a-4817-8788-b5078e54c4c6-service-ca-bundle\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:59.383785 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.383735 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc2b1af9-464a-4817-8788-b5078e54c4c6-metrics-certs\") pod \"router-default-c99488556-jpl8w\" (UID: \"cc2b1af9-464a-4817-8788-b5078e54c4c6\") " pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:59.384881 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.384280 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/826477a9-45f6-4deb-ae6f-d758f49e3d75-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-29tql\" (UID: \"826477a9-45f6-4deb-ae6f-d758f49e3d75\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:59.442454 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.442058 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cxndm"] Jul 21 16:25:59.446994 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:59.446937 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6669ba74_1154_4860_b7d8_8f3482d4255b.slice/crio-4be2758413de48c6bba60aa053b98ca7350915155398ade564ba99e5a2c99c05 WatchSource:0}: Error finding container 4be2758413de48c6bba60aa053b98ca7350915155398ade564ba99e5a2c99c05: Status 404 returned error can't find the container with id 4be2758413de48c6bba60aa053b98ca7350915155398ade564ba99e5a2c99c05 Jul 21 16:25:59.472488 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.471750 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b4888cb99-bz4lx"] Jul 21 16:25:59.474578 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:59.474535 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e60a6c_86b6_4064_9005_2a6751b253bf.slice/crio-2a13593c9fbbc443d749e92b4ec0291a69bf5e897841d99e1451e852dc7dcf4b WatchSource:0}: Error finding container 2a13593c9fbbc443d749e92b4ec0291a69bf5e897841d99e1451e852dc7dcf4b: Status 404 returned error can't find the container with id 2a13593c9fbbc443d749e92b4ec0291a69bf5e897841d99e1451e852dc7dcf4b Jul 21 16:25:59.508977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.508958 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sppfl\"" Jul 21 16:25:59.516488 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.516458 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" Jul 21 16:25:59.523894 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.523871 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj"] Jul 21 16:25:59.526978 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:59.526949 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb369a7f2_8765_4858_9bb0_ab0f52838e21.slice/crio-b85e0d7395b1a154e7132e4742be2c00a74bc6eb8b3e465e4ce8f578bbcd2395 WatchSource:0}: Error finding container b85e0d7395b1a154e7132e4742be2c00a74bc6eb8b3e465e4ce8f578bbcd2395: Status 404 returned error can't find the container with id b85e0d7395b1a154e7132e4742be2c00a74bc6eb8b3e465e4ce8f578bbcd2395 Jul 21 16:25:59.590413 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.590377 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jbwgt\"" Jul 21 16:25:59.598099 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.598060 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:25:59.625981 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.625950 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pfc7p\"" Jul 21 16:25:59.631053 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.631026 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" Jul 21 16:25:59.665964 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.665886 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l"] Jul 21 16:25:59.669586 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:59.669560 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06057afc_c6f1_4735_a59b_20e7ada2f7c9.slice/crio-bcd7811b1d9234ba39e69ee77e2d891b37219733b70b9e2453495a9632e8e75e WatchSource:0}: Error finding container bcd7811b1d9234ba39e69ee77e2d891b37219733b70b9e2453495a9632e8e75e: Status 404 returned error can't find the container with id bcd7811b1d9234ba39e69ee77e2d891b37219733b70b9e2453495a9632e8e75e Jul 21 16:25:59.755492 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.755466 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-c99488556-jpl8w"] Jul 21 16:25:59.758336 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:25:59.758305 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc2b1af9_464a_4817_8788_b5078e54c4c6.slice/crio-20464bc13121b43659166e09b64ea2c79b43d1207f4e00e3d56f18e29f93e5b1 WatchSource:0}: Error finding container 20464bc13121b43659166e09b64ea2c79b43d1207f4e00e3d56f18e29f93e5b1: Status 404 returned error can't find the container with id 20464bc13121b43659166e09b64ea2c79b43d1207f4e00e3d56f18e29f93e5b1 Jul 21 16:25:59.777308 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.777135 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql"] Jul 21 16:25:59.829979 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.829811 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2frkk" event={"ID":"addf81be-facb-4540-8b4c-4960f292deb8","Type":"ContainerStarted","Data":"2758a95d17c3025cfcb1f1fc29c4b1128c5e75ab6540b387f417a8183cae1ca5"} Jul 21 16:25:59.831773 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.831740 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9t8kx" event={"ID":"44dab062-86b8-4849-9d19-75299742a580","Type":"ContainerStarted","Data":"e9283fa7e326b3f1673c49f5d9d4bcbb7929a53c345c0448ace0eb14b3cb0a50"} Jul 21 16:25:59.837681 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.837649 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c99488556-jpl8w" event={"ID":"cc2b1af9-464a-4817-8788-b5078e54c4c6","Type":"ContainerStarted","Data":"20464bc13121b43659166e09b64ea2c79b43d1207f4e00e3d56f18e29f93e5b1"} Jul 21 16:25:59.839477 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.839451 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" event={"ID":"08e60a6c-86b6-4064-9005-2a6751b253bf","Type":"ContainerStarted","Data":"00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb"} Jul 21 16:25:59.839578 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.839495 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" event={"ID":"08e60a6c-86b6-4064-9005-2a6751b253bf","Type":"ContainerStarted","Data":"2a13593c9fbbc443d749e92b4ec0291a69bf5e897841d99e1451e852dc7dcf4b"} Jul 21 16:25:59.839578 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.839543 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:25:59.840714 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.840693 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cxndm" event={"ID":"6669ba74-1154-4860-b7d8-8f3482d4255b","Type":"ContainerStarted","Data":"4be2758413de48c6bba60aa053b98ca7350915155398ade564ba99e5a2c99c05"} Jul 21 16:25:59.842842 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.842802 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bglxt" event={"ID":"64fd93bf-b5a9-4a5e-a97d-29a7d9ea5fbd","Type":"ContainerStarted","Data":"72edba2af8c47e8d50d8be5d9739a93c1a3893dd3fbeb52678164f9b2c72786a"} Jul 21 16:25:59.846383 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.846358 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" event={"ID":"06057afc-c6f1-4735-a59b-20e7ada2f7c9","Type":"ContainerStarted","Data":"bcd7811b1d9234ba39e69ee77e2d891b37219733b70b9e2453495a9632e8e75e"} Jul 21 16:25:59.848206 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.848184 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" event={"ID":"b369a7f2-8765-4858-9bb0-ab0f52838e21","Type":"ContainerStarted","Data":"b85e0d7395b1a154e7132e4742be2c00a74bc6eb8b3e465e4ce8f578bbcd2395"} Jul 21 16:25:59.852272 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.852230 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9t8kx" podStartSLOduration=66.258274359 podStartE2EDuration="1m11.852218347s" totalImagesPullingTime="5.593943988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:59.851736466 +0000 UTC m=+72.890572129" watchObservedRunningTime="2026-07-21 16:25:59.852218347 +0000 UTC m=+72.891053986" Jul 21 16:25:59.874405 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.874292 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" podStartSLOduration=71.874272423 podStartE2EDuration="1m11.874272423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:59.873642263 +0000 UTC m=+72.912477926" watchObservedRunningTime="2026-07-21 16:25:59.874272423 +0000 UTC m=+72.913108087" Jul 21 16:25:59.892629 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:25:59.891600 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bglxt" podStartSLOduration=70.727186011 podStartE2EDuration="1m11.891578659s" totalImagesPullingTime="1.164392648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:25:59.891104151 +0000 UTC m=+72.929939815" watchObservedRunningTime="2026-07-21 16:25:59.891578659 +0000 UTC m=+72.930414323" Jul 21 16:26:00.546876 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.545615 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-mgqrm"] Jul 21 16:26:00.554191 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.554087 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-mgqrm" Jul 21 16:26:00.558397 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.558005 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kbrjz\"" Jul 21 16:26:00.558397 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.558080 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 21 16:26:00.558397 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.558246 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 21 16:26:00.574669 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.574637 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-mgqrm"] Jul 21 16:26:00.592123 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.592087 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2gfv\" (UniqueName: \"kubernetes.io/projected/59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9-kube-api-access-l2gfv\") pod \"downloads-5bc67dcd46-mgqrm\" (UID: \"59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9\") " pod="openshift-console/downloads-5bc67dcd46-mgqrm" Jul 21 16:26:00.693108 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.692653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2gfv\" (UniqueName: \"kubernetes.io/projected/59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9-kube-api-access-l2gfv\") pod \"downloads-5bc67dcd46-mgqrm\" (UID: \"59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9\") " pod="openshift-console/downloads-5bc67dcd46-mgqrm" Jul 21 16:26:00.708210 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.708146 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2gfv\" (UniqueName: \"kubernetes.io/projected/59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9-kube-api-access-l2gfv\") pod \"downloads-5bc67dcd46-mgqrm\" (UID: \"59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9\") " pod="openshift-console/downloads-5bc67dcd46-mgqrm" Jul 21 16:26:00.860621 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.860534 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-c99488556-jpl8w" event={"ID":"cc2b1af9-464a-4817-8788-b5078e54c4c6","Type":"ContainerStarted","Data":"089eeac139307c2566ca171a352eb66cb37c24d5598f72e3a9f030f0af4cb058"} Jul 21 16:26:00.863074 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.863033 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" event={"ID":"826477a9-45f6-4deb-ae6f-d758f49e3d75","Type":"ContainerStarted","Data":"ba45672982f4333f3cee7b67e84b03fbeb40f7a8244e90ebaf6175d5154c42e3"} Jul 21 16:26:00.874236 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:00.874202 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-mgqrm" Jul 21 16:26:01.599059 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:01.599013 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:26:01.601900 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:01.601876 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:26:01.638111 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:01.638042 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-c99488556-jpl8w" podStartSLOduration=34.638019075 podStartE2EDuration="34.638019075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:00.897432304 +0000 UTC m=+73.936267969" watchObservedRunningTime="2026-07-21 16:26:01.638019075 +0000 UTC m=+74.676854739" Jul 21 16:26:01.868435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:01.868354 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:26:01.869913 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:01.869889 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-c99488556-jpl8w" Jul 21 16:26:06.438083 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.438059 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-mgqrm"] Jul 21 16:26:06.439068 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:26:06.439047 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59fdf7f1_4d2e_4cad_93c0_01eb08ae20b9.slice/crio-428904d60556530ebf9f14ba410cdb1479f4af5c223dc6e503dee820f755a77e WatchSource:0}: Error finding container 428904d60556530ebf9f14ba410cdb1479f4af5c223dc6e503dee820f755a77e: Status 404 returned error can't find the container with id 428904d60556530ebf9f14ba410cdb1479f4af5c223dc6e503dee820f755a77e Jul 21 16:26:06.879891 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.879812 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw"] Jul 21 16:26:06.899420 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899391 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw"] Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899432 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cxndm" event={"ID":"6669ba74-1154-4860-b7d8-8f3482d4255b","Type":"ContainerStarted","Data":"44ddd7a9a3080b0736e5c076c3174d9531476f7a571fc739a69f973c97de90d3"} Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899471 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cxndm" Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899493 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cxndm" event={"ID":"6669ba74-1154-4860-b7d8-8f3482d4255b","Type":"ContainerStarted","Data":"356e0bd6bf0a960e1714dc8fbfe8cb34bad92f6f5d92c63080ba18ad0860c200"} Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899516 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" event={"ID":"826477a9-45f6-4deb-ae6f-d758f49e3d75","Type":"ContainerStarted","Data":"559f1bc38ad486a0228a3805a269031466cafbffeab9df956edf6a92faef910f"} Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899542 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" event={"ID":"826477a9-45f6-4deb-ae6f-d758f49e3d75","Type":"ContainerStarted","Data":"66c921664536cfb09feadcb3b05fcb8b777afdb9f13eb5a442249bcf84d68e98"} Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" event={"ID":"06057afc-c6f1-4735-a59b-20e7ada2f7c9","Type":"ContainerStarted","Data":"5811b61f9347024689c8d24e2496003a6086d7b40558c4f338b90328c0eed32d"} Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899585 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" event={"ID":"b369a7f2-8765-4858-9bb0-ab0f52838e21","Type":"ContainerStarted","Data":"5c2cc1bcbe271247594c57188378b3a3afa3eb891c8f6c06892dfee5ab789c67"} Jul 21 16:26:06.899597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899584 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:06.899985 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.899761 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-mgqrm" event={"ID":"59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9","Type":"ContainerStarted","Data":"428904d60556530ebf9f14ba410cdb1479f4af5c223dc6e503dee820f755a77e"} Jul 21 16:26:06.902134 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.902106 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2frkk" event={"ID":"addf81be-facb-4540-8b4c-4960f292deb8","Type":"ContainerStarted","Data":"35fb4ac93864018c24f66173028169fe76e69404c83a4e0f17f8c3a928826c37"} Jul 21 16:26:06.903521 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.903500 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 21 16:26:06.903636 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.903601 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-2s4w2\"" Jul 21 16:26:06.942882 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.942781 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ee472008-8795-4ff5-adea-fa2adaeb187e-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-hljbw\" (UID: \"ee472008-8795-4ff5-adea-fa2adaeb187e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:06.958086 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.958030 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cxndm" podStartSLOduration=37.112801887 podStartE2EDuration="43.958009639s" totalImagesPullingTime="6.845207752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:06.926732085 +0000 UTC m=+79.965567747" watchObservedRunningTime="2026-07-21 16:26:06.958009639 +0000 UTC m=+79.996845302" Jul 21 16:26:06.958229 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.958169 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jxxfj" podStartSLOduration=33.190547367 podStartE2EDuration="39.958159051s" totalImagesPullingTime="6.767611684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:06.956573601 +0000 UTC m=+79.995409264" watchObservedRunningTime="2026-07-21 16:26:06.958159051 +0000 UTC m=+79.996994715" Jul 21 16:26:06.976357 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.976304 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4q94l" podStartSLOduration=33.354117128 podStartE2EDuration="39.976283464s" totalImagesPullingTime="6.622166336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:06.975657084 +0000 UTC m=+80.014492750" watchObservedRunningTime="2026-07-21 16:26:06.976283464 +0000 UTC m=+80.015119128" Jul 21 16:26:06.999738 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:06.999666 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-29tql" podStartSLOduration=33.535924681 podStartE2EDuration="39.999643626s" totalImagesPullingTime="6.463718945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:25:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:06.998977166 +0000 UTC m=+80.037812830" watchObservedRunningTime="2026-07-21 16:26:06.999643626 +0000 UTC m=+80.038479290" Jul 21 16:26:07.043594 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:07.043554 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ee472008-8795-4ff5-adea-fa2adaeb187e-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-hljbw\" (UID: \"ee472008-8795-4ff5-adea-fa2adaeb187e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:07.043763 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:26:07.043741 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 21 16:26:07.043838 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:26:07.043806 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee472008-8795-4ff5-adea-fa2adaeb187e-tls-certificates podName:ee472008-8795-4ff5-adea-fa2adaeb187e nodeName:}" failed. No retries permitted until 2026-07-21 16:26:07.543785126 +0000 UTC m=+80.582620774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/ee472008-8795-4ff5-adea-fa2adaeb187e-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-hljbw" (UID: "ee472008-8795-4ff5-adea-fa2adaeb187e") : secret "prometheus-operator-admission-webhook-tls" not found Jul 21 16:26:07.548295 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:07.548254 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ee472008-8795-4ff5-adea-fa2adaeb187e-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-hljbw\" (UID: \"ee472008-8795-4ff5-adea-fa2adaeb187e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:07.551206 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:07.551177 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ee472008-8795-4ff5-adea-fa2adaeb187e-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-hljbw\" (UID: \"ee472008-8795-4ff5-adea-fa2adaeb187e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:07.810763 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:07.810676 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:07.994984 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:07.994952 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw"] Jul 21 16:26:07.998174 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:26:07.998143 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee472008_8795_4ff5_adea_fa2adaeb187e.slice/crio-1ddb65c33d4c066673cb704972401e9e8978656836db82fda4e8393342173191 WatchSource:0}: Error finding container 1ddb65c33d4c066673cb704972401e9e8978656836db82fda4e8393342173191: Status 404 returned error can't find the container with id 1ddb65c33d4c066673cb704972401e9e8978656836db82fda4e8393342173191 Jul 21 16:26:08.917318 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:08.917270 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" event={"ID":"ee472008-8795-4ff5-adea-fa2adaeb187e","Type":"ContainerStarted","Data":"1ddb65c33d4c066673cb704972401e9e8978656836db82fda4e8393342173191"} Jul 21 16:26:11.938598 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:11.938557 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2frkk" event={"ID":"addf81be-facb-4540-8b4c-4960f292deb8","Type":"ContainerStarted","Data":"21bda79d4371d913ea7dd404fed6df3c7ae45608855252bbd66a9cfa8958fa9a"} Jul 21 16:26:11.940225 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:11.940192 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" event={"ID":"ee472008-8795-4ff5-adea-fa2adaeb187e","Type":"ContainerStarted","Data":"4593d219df564a25486700a2a1f9196a91f18c1c718a15e51652a21388040e9d"} Jul 21 16:26:11.940431 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:11.940412 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:11.945717 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:11.945694 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" Jul 21 16:26:11.958417 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:11.958367 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2frkk" podStartSLOduration=17.677721277 podStartE2EDuration="35.958352149s" totalImagesPullingTime="18.280630872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 16:25:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:11.957151948 +0000 UTC m=+84.995987611" watchObservedRunningTime="2026-07-21 16:26:11.958352149 +0000 UTC m=+84.997187843" Jul 21 16:26:11.974435 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:11.974388 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-hljbw" podStartSLOduration=2.924902732 podStartE2EDuration="5.974372259s" totalImagesPullingTime="3.049469527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:11.973809165 +0000 UTC m=+85.012644827" watchObservedRunningTime="2026-07-21 16:26:11.974372259 +0000 UTC m=+85.013207921" Jul 21 16:26:12.955166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.955135 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6"] Jul 21 16:26:12.957693 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.957665 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:12.963268 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.963245 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 21 16:26:12.963407 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.963294 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-zpcvc\"" Jul 21 16:26:12.963521 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.963498 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 21 16:26:12.964767 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.964746 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 16:26:12.980172 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.980144 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6"] Jul 21 16:26:12.993572 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.993533 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:12.993725 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.993636 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcc4k\" (UniqueName: \"kubernetes.io/projected/7a93b7bc-6795-47d6-a337-9c7504954d09-kube-api-access-bcc4k\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:12.993725 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.993694 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:12.993812 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:12.993798 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a93b7bc-6795-47d6-a337-9c7504954d09-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.094880 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.094840 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a93b7bc-6795-47d6-a337-9c7504954d09-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.095103 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.095005 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.095103 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.095055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcc4k\" (UniqueName: \"kubernetes.io/projected/7a93b7bc-6795-47d6-a337-9c7504954d09-kube-api-access-bcc4k\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.095103 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.095095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.095222 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:26:13.095143 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 21 16:26:13.095222 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:26:13.095211 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-tls podName:7a93b7bc-6795-47d6-a337-9c7504954d09 nodeName:}" failed. No retries permitted until 2026-07-21 16:26:13.595190493 +0000 UTC m=+86.634026150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-m8ng6" (UID: "7a93b7bc-6795-47d6-a337-9c7504954d09") : secret "prometheus-operator-tls" not found Jul 21 16:26:13.095694 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.095658 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a93b7bc-6795-47d6-a337-9c7504954d09-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.097601 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.097579 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.104944 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.104900 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcc4k\" (UniqueName: \"kubernetes.io/projected/7a93b7bc-6795-47d6-a337-9c7504954d09-kube-api-access-bcc4k\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.600157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.600121 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.603383 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.603349 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a93b7bc-6795-47d6-a337-9c7504954d09-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-m8ng6\" (UID: \"7a93b7bc-6795-47d6-a337-9c7504954d09\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:13.869560 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:13.869462 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" Jul 21 16:26:14.028808 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:14.028773 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6"] Jul 21 16:26:14.030360 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:26:14.030332 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a93b7bc_6795_47d6_a337_9c7504954d09.slice/crio-281bc8bd9b208f49865bd2dad918ac8c5c0755f440335fe48d85005d130cbb75 WatchSource:0}: Error finding container 281bc8bd9b208f49865bd2dad918ac8c5c0755f440335fe48d85005d130cbb75: Status 404 returned error can't find the container with id 281bc8bd9b208f49865bd2dad918ac8c5c0755f440335fe48d85005d130cbb75 Jul 21 16:26:14.956603 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:14.956564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" event={"ID":"7a93b7bc-6795-47d6-a337-9c7504954d09","Type":"ContainerStarted","Data":"281bc8bd9b208f49865bd2dad918ac8c5c0755f440335fe48d85005d130cbb75"} Jul 21 16:26:15.965437 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:15.965300 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" event={"ID":"7a93b7bc-6795-47d6-a337-9c7504954d09","Type":"ContainerStarted","Data":"3a985ee164ba30fbbd002634acd721c80e39289e83d0f87d05b2978019f76b05"} Jul 21 16:26:15.965437 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:15.965408 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" event={"ID":"7a93b7bc-6795-47d6-a337-9c7504954d09","Type":"ContainerStarted","Data":"0cb540acd7122c2afa89912961c0595d77940bf705ccb227326c4c6b7cb56111"} Jul 21 16:26:15.985303 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:15.985250 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-m8ng6" podStartSLOduration=2.305740834 podStartE2EDuration="3.985232073s" totalImagesPullingTime="1.679491239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:15.983374437 +0000 UTC m=+89.022210101" watchObservedRunningTime="2026-07-21 16:26:15.985232073 +0000 UTC m=+89.024067734" Jul 21 16:26:17.920543 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:17.920509 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cxndm" Jul 21 16:26:18.378579 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.378542 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h642n"] Jul 21 16:26:18.382106 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.382074 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-jqqmt"] Jul 21 16:26:18.382257 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.382201 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.384576 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.384556 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.385685 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.385665 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 16:26:18.385797 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.385707 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 16:26:18.386058 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.386036 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 16:26:18.386359 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.386341 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-42mkg\"" Jul 21 16:26:18.387565 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.387378 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 21 16:26:18.387718 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.387686 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 21 16:26:18.387895 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.387879 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-s77f2\"" Jul 21 16:26:18.387949 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.387894 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 21 16:26:18.402028 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.402001 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-jqqmt"] Jul 21 16:26:18.439084 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439041 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a7575720-bd03-4d30-85a9-a05da5073932-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.439084 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439089 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.439301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439147 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7575720-bd03-4d30-85a9-a05da5073932-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.439301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439169 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-textfile\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439213 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-root\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439237 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-tls\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439271 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-accelerators-collector-config\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439297 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zdw2\" (UniqueName: \"kubernetes.io/projected/a7575720-bd03-4d30-85a9-a05da5073932-kube-api-access-8zdw2\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.439639 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439359 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.439639 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.439639 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h762n\" (UniqueName: \"kubernetes.io/projected/74df7180-83e0-4993-9111-12fb70e0d1b4-kube-api-access-h762n\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439798 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439644 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-sys\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439798 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439673 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-wtmp\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.439798 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.439756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74df7180-83e0-4993-9111-12fb70e0d1b4-metrics-client-ca\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.540762 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7575720-bd03-4d30-85a9-a05da5073932-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540773 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-textfile\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540813 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-root\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540866 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-tls\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-accelerators-collector-config\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540901 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-root\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540935 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zdw2\" (UniqueName: \"kubernetes.io/projected/a7575720-bd03-4d30-85a9-a05da5073932-kube-api-access-8zdw2\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.540972 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.540995 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:26:18.541014 2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541022 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h762n\" (UniqueName: \"kubernetes.io/projected/74df7180-83e0-4993-9111-12fb70e0d1b4-kube-api-access-h762n\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-sys\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:26:18.541076 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-tls podName:74df7180-83e0-4993-9111-12fb70e0d1b4 nodeName:}" failed. No retries permitted until 2026-07-21 16:26:19.041054768 +0000 UTC m=+92.079890412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-tls") pod "node-exporter-h642n" (UID: "74df7180-83e0-4993-9111-12fb70e0d1b4") : secret "node-exporter-tls" not found Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541101 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-sys\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541113 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-wtmp\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541140 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74df7180-83e0-4993-9111-12fb70e0d1b4-metrics-client-ca\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541168 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a7575720-bd03-4d30-85a9-a05da5073932-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541170 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-textfile\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.541376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541232 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.541969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541419 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-wtmp\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.541969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541503 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/a7575720-bd03-4d30-85a9-a05da5073932-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.541969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541525 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a7575720-bd03-4d30-85a9-a05da5073932-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.541969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.541641 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-accelerators-collector-config\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.542164 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.542031 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74df7180-83e0-4993-9111-12fb70e0d1b4-metrics-client-ca\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.542164 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.542144 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.543936 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.543917 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.544046 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.543994 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.544425 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.544404 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7575720-bd03-4d30-85a9-a05da5073932-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.550197 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.550173 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zdw2\" (UniqueName: \"kubernetes.io/projected/a7575720-bd03-4d30-85a9-a05da5073932-kube-api-access-8zdw2\") pod \"kube-state-metrics-56c5594674-jqqmt\" (UID: \"a7575720-bd03-4d30-85a9-a05da5073932\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:18.559442 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.557440 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h762n\" (UniqueName: \"kubernetes.io/projected/74df7180-83e0-4993-9111-12fb70e0d1b4-kube-api-access-h762n\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:18.702189 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:18.702101 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" Jul 21 16:26:19.046385 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:19.046353 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-tls\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:19.049050 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:19.049023 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/74df7180-83e0-4993-9111-12fb70e0d1b4-node-exporter-tls\") pod \"node-exporter-h642n\" (UID: \"74df7180-83e0-4993-9111-12fb70e0d1b4\") " pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:19.295532 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:19.295491 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h642n" Jul 21 16:26:20.868544 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:20.868506 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:26:22.330982 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.330945 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk"] Jul 21 16:26:22.479000 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.478920 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk"] Jul 21 16:26:22.479177 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.479049 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.482628 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.482339 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-55hdn\"" Jul 21 16:26:22.482628 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.482358 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 21 16:26:22.482628 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.482359 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 21 16:26:22.482628 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.482402 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 21 16:26:22.482628 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.482339 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 21 16:26:22.482628 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.482578 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 21 16:26:22.483034 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.482758 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bmsh8qrpr080g\"" Jul 21 16:26:22.571584 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.571547 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b4888cb99-bz4lx"] Jul 21 16:26:22.580681 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.580645 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.580914 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.580888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0156c2-594c-462c-ac56-50caf968fe96-metrics-client-ca\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.581175 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.581114 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.581175 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.581158 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-tls\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.581315 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.581193 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.581315 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.581258 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jb5s\" (UniqueName: \"kubernetes.io/projected/9b0156c2-594c-462c-ac56-50caf968fe96-kube-api-access-8jb5s\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.581315 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.581289 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.581438 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.581388 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-grpc-tls\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.682632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682591 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.682833 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682649 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-tls\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.682833 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682682 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.682833 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682715 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jb5s\" (UniqueName: \"kubernetes.io/projected/9b0156c2-594c-462c-ac56-50caf968fe96-kube-api-access-8jb5s\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.682833 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682742 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.682833 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682801 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-grpc-tls\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.683095 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.683095 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.682914 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0156c2-594c-462c-ac56-50caf968fe96-metrics-client-ca\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.683706 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.683677 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0156c2-594c-462c-ac56-50caf968fe96-metrics-client-ca\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.685998 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.685947 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-tls\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.686154 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.686124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.686247 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.686212 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.686614 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.686587 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.687002 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.686954 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-grpc-tls\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.687130 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.687109 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0156c2-594c-462c-ac56-50caf968fe96-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.700178 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.700146 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jb5s\" (UniqueName: \"kubernetes.io/projected/9b0156c2-594c-462c-ac56-50caf968fe96-kube-api-access-8jb5s\") pod \"thanos-querier-7fbf7f598c-mnmnk\" (UID: \"9b0156c2-594c-462c-ac56-50caf968fe96\") " pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:22.790678 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:22.790644 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:24.387198 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:26:24.387150 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74df7180_83e0_4993_9111_12fb70e0d1b4.slice/crio-e5e9deff4da819ab448e4fb5f4fee5068c25774d24be6a63869fa108173d0006 WatchSource:0}: Error finding container e5e9deff4da819ab448e4fb5f4fee5068c25774d24be6a63869fa108173d0006: Status 404 returned error can't find the container with id e5e9deff4da819ab448e4fb5f4fee5068c25774d24be6a63869fa108173d0006 Jul 21 16:26:24.545054 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:24.544966 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-jqqmt"] Jul 21 16:26:24.572746 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:24.572649 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk"] Jul 21 16:26:24.573110 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:26:24.573086 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7575720_bd03_4d30_85a9_a05da5073932.slice/crio-aceba5784ac7e458b019c508fa08cec453d74e01fa2703828e55947faf0d0f93 WatchSource:0}: Error finding container aceba5784ac7e458b019c508fa08cec453d74e01fa2703828e55947faf0d0f93: Status 404 returned error can't find the container with id aceba5784ac7e458b019c508fa08cec453d74e01fa2703828e55947faf0d0f93 Jul 21 16:26:24.575204 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:26:24.575137 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0156c2_594c_462c_ac56_50caf968fe96.slice/crio-34b0cb9b27ae6b3afb6d42474229546aec0959c4e1e8aaebd941efa78194cbb2 WatchSource:0}: Error finding container 34b0cb9b27ae6b3afb6d42474229546aec0959c4e1e8aaebd941efa78194cbb2: Status 404 returned error can't find the container with id 34b0cb9b27ae6b3afb6d42474229546aec0959c4e1e8aaebd941efa78194cbb2 Jul 21 16:26:25.015902 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:25.015866 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-mgqrm" event={"ID":"59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9","Type":"ContainerStarted","Data":"f3a096172435b97455f9ec7f1416f176c004c1d6781d7d399f26f7acaba82f87"} Jul 21 16:26:25.016417 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:25.016388 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-mgqrm" Jul 21 16:26:25.018181 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:25.017879 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h642n" event={"ID":"74df7180-83e0-4993-9111-12fb70e0d1b4","Type":"ContainerStarted","Data":"e5e9deff4da819ab448e4fb5f4fee5068c25774d24be6a63869fa108173d0006"} Jul 21 16:26:25.019565 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:25.019542 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" event={"ID":"a7575720-bd03-4d30-85a9-a05da5073932","Type":"ContainerStarted","Data":"aceba5784ac7e458b019c508fa08cec453d74e01fa2703828e55947faf0d0f93"} Jul 21 16:26:25.021083 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:25.021063 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" event={"ID":"9b0156c2-594c-462c-ac56-50caf968fe96","Type":"ContainerStarted","Data":"34b0cb9b27ae6b3afb6d42474229546aec0959c4e1e8aaebd941efa78194cbb2"} Jul 21 16:26:25.022289 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:25.022268 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-mgqrm" Jul 21 16:26:25.035694 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:25.035624 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-mgqrm" podStartSLOduration=6.834197197 podStartE2EDuration="25.035603319s" totalImagesPullingTime="18.201406122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:26:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:25.034155303 +0000 UTC m=+98.072990969" watchObservedRunningTime="2026-07-21 16:26:25.035603319 +0000 UTC m=+98.074438982" Jul 21 16:26:26.814145 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:26.814108 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6r7qk" Jul 21 16:26:29.063876 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.063812 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" event={"ID":"a7575720-bd03-4d30-85a9-a05da5073932","Type":"ContainerStarted","Data":"9fcda2e0c4eccb20d93cf2f26b275f6f14aa3448d336dd0344e31a9aaa6ffa11"} Jul 21 16:26:29.064459 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.064424 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" event={"ID":"a7575720-bd03-4d30-85a9-a05da5073932","Type":"ContainerStarted","Data":"69b9468aeec0a375b2adc212297a69d139a003ef2420b168dfa25ea949788cf3"} Jul 21 16:26:29.064584 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.064474 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" event={"ID":"a7575720-bd03-4d30-85a9-a05da5073932","Type":"ContainerStarted","Data":"9e554b12d48b5cdd6ef2ce2b8c3b385c53d7cf7c2bbb7fd89cd778981466482f"} Jul 21 16:26:29.067637 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.067604 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" event={"ID":"9b0156c2-594c-462c-ac56-50caf968fe96","Type":"ContainerStarted","Data":"31172c0d8b21eb5b5289d80b4f760d772f79f938961c0196ea981244fd5ab827"} Jul 21 16:26:29.067809 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.067684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" event={"ID":"9b0156c2-594c-462c-ac56-50caf968fe96","Type":"ContainerStarted","Data":"2cea755520b968e75242cf1fe0d90f36bc8a64d48eb3cf31ac7bf720d365e852"} Jul 21 16:26:29.067809 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.067731 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" event={"ID":"9b0156c2-594c-462c-ac56-50caf968fe96","Type":"ContainerStarted","Data":"7eb1513eb197b34ea2c83a66e1e0e3ff094f468f1306a57d90fc9125d89dc45e"} Jul 21 16:26:29.069781 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.069719 2557 generic.go:356] "Generic (PLEG): container finished" podID="74df7180-83e0-4993-9111-12fb70e0d1b4" containerID="e656162f5f7fe9699b4ffdeaa2cb906995cae97e42bdd3d53528e869dc8a188b" exitCode=0 Jul 21 16:26:29.069905 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.069779 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h642n" event={"ID":"74df7180-83e0-4993-9111-12fb70e0d1b4","Type":"ContainerDied","Data":"e656162f5f7fe9699b4ffdeaa2cb906995cae97e42bdd3d53528e869dc8a188b"} Jul 21 16:26:29.087376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:29.087311 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-jqqmt" podStartSLOduration=7.6303964650000005 podStartE2EDuration="11.087293396s" totalImagesPullingTime="3.456896931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:29.086223638 +0000 UTC m=+102.125059301" watchObservedRunningTime="2026-07-21 16:26:29.087293396 +0000 UTC m=+102.126129059" Jul 21 16:26:30.078262 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:30.078221 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" event={"ID":"9b0156c2-594c-462c-ac56-50caf968fe96","Type":"ContainerStarted","Data":"bf6407f0294188f9647c9bc3c13d3cb8e4e3bdb946a6081f075f6174ef465bd3"} Jul 21 16:26:30.080986 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:30.080948 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h642n" event={"ID":"74df7180-83e0-4993-9111-12fb70e0d1b4","Type":"ContainerStarted","Data":"b0f571c8b45f2adc508253afc48b7dc282139991cbf25d6ab95b5d241f1cf9a5"} Jul 21 16:26:30.081105 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:30.081008 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h642n" event={"ID":"74df7180-83e0-4993-9111-12fb70e0d1b4","Type":"ContainerStarted","Data":"a82d8165917ddd3082a12072fb6744235c043f6ecb209161647ae62843b9ceba"} Jul 21 16:26:30.105418 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:30.104728 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h642n" podStartSLOduration=8.328750278 podStartE2EDuration="12.104707272s" totalImagesPullingTime="3.652072097s" totalInitContainerRuntime="123.884897ms" isStatefulPod=true podCreationTimestamp="2026-07-21 16:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:30.102257531 +0000 UTC m=+103.141093216" watchObservedRunningTime="2026-07-21 16:26:30.104707272 +0000 UTC m=+103.143542936" Jul 21 16:26:31.090145 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:31.090102 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" event={"ID":"9b0156c2-594c-462c-ac56-50caf968fe96","Type":"ContainerStarted","Data":"2355c2d72b6a50042850d0a2f792125a2ead2d61963ca3f8b2a014e26c5cb247"} Jul 21 16:26:31.090707 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:31.090192 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" event={"ID":"9b0156c2-594c-462c-ac56-50caf968fe96","Type":"ContainerStarted","Data":"825467b12627755e81ea2507f15acdc807bc111f28fae1b0bcc04f1e7f863e22"} Jul 21 16:26:31.117891 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:31.117812 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" podStartSLOduration=4.190817923 podStartE2EDuration="9.117788745s" totalImagesPullingTime="4.926970822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:26:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:26:31.114792465 +0000 UTC m=+104.153628141" watchObservedRunningTime="2026-07-21 16:26:31.117788745 +0000 UTC m=+104.156624408" Jul 21 16:26:32.094632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:32.094594 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:33.104621 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:33.104589 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7fbf7f598c-mnmnk" Jul 21 16:26:46.153521 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:46.153429 2557 generic.go:356] "Generic (PLEG): container finished" podID="e5edd2df-971a-4809-b26d-51110cac4ead" containerID="bcc1615334bd94d0d6095003b4203a886bc24d8577c91657781f7fe18672730b" exitCode=0 Jul 21 16:26:46.153521 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:46.153480 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" event={"ID":"e5edd2df-971a-4809-b26d-51110cac4ead","Type":"ContainerDied","Data":"bcc1615334bd94d0d6095003b4203a886bc24d8577c91657781f7fe18672730b"} Jul 21 16:26:46.153980 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:46.153793 2557 scope.go:117] "RemoveContainer" containerID="bcc1615334bd94d0d6095003b4203a886bc24d8577c91657781f7fe18672730b" Jul 21 16:26:47.160067 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:47.160029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-qknkl" event={"ID":"e5edd2df-971a-4809-b26d-51110cac4ead","Type":"ContainerStarted","Data":"b299c7fcc620f0362e85f76451b45fc81075f33f49cd9ba2ae22899fb7f21f9d"} Jul 21 16:26:47.599919 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:47.599873 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" podUID="08e60a6c-86b6-4064-9005-2a6751b253bf" containerName="registry" containerID="cri-o://00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb" gracePeriod=30 Jul 21 16:26:47.880396 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:47.880370 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:26:48.020847 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.020788 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-certificates\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.020864 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.020906 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-image-registry-private-configuration\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.020940 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08e60a6c-86b6-4064-9005-2a6751b253bf-ca-trust-extracted\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.020980 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-trusted-ca\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.021018 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-bound-sa-token\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021278 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.021058 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-installation-pull-secrets\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021278 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.021086 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5d5q7\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-kube-api-access-5d5q7\") pod \"08e60a6c-86b6-4064-9005-2a6751b253bf\" (UID: \"08e60a6c-86b6-4064-9005-2a6751b253bf\") " Jul 21 16:26:48.021384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.021312 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:26:48.021538 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.021474 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:26:48.023563 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.023529 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 16:26:48.023563 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.023553 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:26:48.023734 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.023684 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:26:48.023856 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.023835 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-kube-api-access-5d5q7" (OuterVolumeSpecName: "kube-api-access-5d5q7") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "kube-api-access-5d5q7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:26:48.023856 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.023842 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 16:26:48.030371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.030348 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08e60a6c-86b6-4064-9005-2a6751b253bf-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "08e60a6c-86b6-4064-9005-2a6751b253bf" (UID: "08e60a6c-86b6-4064-9005-2a6751b253bf"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122555 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-trusted-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122582 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-bound-sa-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122591 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-installation-pull-secrets\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122600 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5d5q7\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-kube-api-access-5d5q7\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122609 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-certificates\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122617 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/08e60a6c-86b6-4064-9005-2a6751b253bf-registry-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122627 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/08e60a6c-86b6-4064-9005-2a6751b253bf-image-registry-private-configuration\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.122632 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.122636 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/08e60a6c-86b6-4064-9005-2a6751b253bf-ca-trust-extracted\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:26:48.164869 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.164836 2557 generic.go:356] "Generic (PLEG): container finished" podID="08e60a6c-86b6-4064-9005-2a6751b253bf" containerID="00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb" exitCode=0 Jul 21 16:26:48.165244 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.164878 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" Jul 21 16:26:48.165244 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.164914 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" event={"ID":"08e60a6c-86b6-4064-9005-2a6751b253bf","Type":"ContainerDied","Data":"00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb"} Jul 21 16:26:48.165244 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.164963 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b4888cb99-bz4lx" event={"ID":"08e60a6c-86b6-4064-9005-2a6751b253bf","Type":"ContainerDied","Data":"2a13593c9fbbc443d749e92b4ec0291a69bf5e897841d99e1451e852dc7dcf4b"} Jul 21 16:26:48.165244 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.164984 2557 scope.go:117] "RemoveContainer" containerID="00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb" Jul 21 16:26:48.173749 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.173732 2557 scope.go:117] "RemoveContainer" containerID="00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb" Jul 21 16:26:48.174018 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:26:48.173996 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb\": container with ID starting with 00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb not found: ID does not exist" containerID="00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb" Jul 21 16:26:48.174097 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.174030 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb"} err="failed to get container status \"00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb\": rpc error: code = NotFound desc = could not find container \"00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb\": container with ID starting with 00cb664ef506fa91a83854f76cec9257b4aeb94b9c69b58e7b1ee4d702e976fb not found: ID does not exist" Jul 21 16:26:48.189958 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.188689 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b4888cb99-bz4lx"] Jul 21 16:26:48.193204 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:48.193180 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b4888cb99-bz4lx"] Jul 21 16:26:49.488036 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:49.487993 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08e60a6c-86b6-4064-9005-2a6751b253bf" path="/var/lib/kubelet/pods/08e60a6c-86b6-4064-9005-2a6751b253bf/volumes" Jul 21 16:26:51.179721 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:51.179634 2557 generic.go:356] "Generic (PLEG): container finished" podID="38bb2b9c-3136-418d-a21d-1d2759954f76" containerID="f2c2d2808bef0a3ec0ca5ef7c9dfdcaf8637fb6d855a33a151e77b93b027acdd" exitCode=0 Jul 21 16:26:51.179721 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:51.179682 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" event={"ID":"38bb2b9c-3136-418d-a21d-1d2759954f76","Type":"ContainerDied","Data":"f2c2d2808bef0a3ec0ca5ef7c9dfdcaf8637fb6d855a33a151e77b93b027acdd"} Jul 21 16:26:51.180136 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:51.180022 2557 scope.go:117] "RemoveContainer" containerID="f2c2d2808bef0a3ec0ca5ef7c9dfdcaf8637fb6d855a33a151e77b93b027acdd" Jul 21 16:26:52.184863 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:26:52.184799 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-fbqg8" event={"ID":"38bb2b9c-3136-418d-a21d-1d2759954f76","Type":"ContainerStarted","Data":"5dbe11eb64346521d83dce8f3f20de29e9af1e67d3bd7985439c7b96144f7366"} Jul 21 16:27:01.226229 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:27:01.226196 2557 generic.go:356] "Generic (PLEG): container finished" podID="23d93917-4b64-4728-a183-f39e4530367a" containerID="28caa750c527cf26e924c546890a7a310929e3954cc10c70f43e463bc1b01c53" exitCode=0 Jul 21 16:27:01.226604 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:27:01.226275 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-h2cdv" event={"ID":"23d93917-4b64-4728-a183-f39e4530367a","Type":"ContainerDied","Data":"28caa750c527cf26e924c546890a7a310929e3954cc10c70f43e463bc1b01c53"} Jul 21 16:27:01.226645 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:27:01.226618 2557 scope.go:117] "RemoveContainer" containerID="28caa750c527cf26e924c546890a7a310929e3954cc10c70f43e463bc1b01c53" Jul 21 16:27:02.232028 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:27:02.231991 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-h2cdv" event={"ID":"23d93917-4b64-4728-a183-f39e4530367a","Type":"ContainerStarted","Data":"345ee79f352139eb9b2f9e17ed757a763da2c788928b56b9c8fa5c2983fe93a2"} Jul 21 16:29:09.752327 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.752249 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5"] Jul 21 16:29:09.752864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.752562 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08e60a6c-86b6-4064-9005-2a6751b253bf" containerName="registry" Jul 21 16:29:09.752864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.752579 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="08e60a6c-86b6-4064-9005-2a6751b253bf" containerName="registry" Jul 21 16:29:09.752864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.752695 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="08e60a6c-86b6-4064-9005-2a6751b253bf" containerName="registry" Jul 21 16:29:09.754853 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.754837 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:09.757725 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.757707 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xtvfw\"" Jul 21 16:29:09.758075 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.758060 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 21 16:29:09.758781 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.758763 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6363a48-8a18-40a5-a052-54a5cba03f8b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqfn5\" (UID: \"b6363a48-8a18-40a5-a052-54a5cba03f8b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:09.758880 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.758802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pdl5\" (UniqueName: \"kubernetes.io/projected/b6363a48-8a18-40a5-a052-54a5cba03f8b-kube-api-access-6pdl5\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqfn5\" (UID: \"b6363a48-8a18-40a5-a052-54a5cba03f8b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:09.759679 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.759662 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 21 16:29:09.774123 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.774093 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5"] Jul 21 16:29:09.859425 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.859397 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pdl5\" (UniqueName: \"kubernetes.io/projected/b6363a48-8a18-40a5-a052-54a5cba03f8b-kube-api-access-6pdl5\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqfn5\" (UID: \"b6363a48-8a18-40a5-a052-54a5cba03f8b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:09.859588 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.859458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6363a48-8a18-40a5-a052-54a5cba03f8b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqfn5\" (UID: \"b6363a48-8a18-40a5-a052-54a5cba03f8b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:09.859762 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.859747 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6363a48-8a18-40a5-a052-54a5cba03f8b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqfn5\" (UID: \"b6363a48-8a18-40a5-a052-54a5cba03f8b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:09.868667 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:09.868645 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pdl5\" (UniqueName: \"kubernetes.io/projected/b6363a48-8a18-40a5-a052-54a5cba03f8b-kube-api-access-6pdl5\") pod \"cert-manager-operator-controller-manager-55c444bf86-zqfn5\" (UID: \"b6363a48-8a18-40a5-a052-54a5cba03f8b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:10.064014 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:10.063921 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" Jul 21 16:29:10.193089 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:10.193064 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5"] Jul 21 16:29:10.195728 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:29:10.195704 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6363a48_8a18_40a5_a052_54a5cba03f8b.slice/crio-5fb38e1435f15ece9bbed786dc06d6b53137e1f90b846adb99396c02fab60852 WatchSource:0}: Error finding container 5fb38e1435f15ece9bbed786dc06d6b53137e1f90b846adb99396c02fab60852: Status 404 returned error can't find the container with id 5fb38e1435f15ece9bbed786dc06d6b53137e1f90b846adb99396c02fab60852 Jul 21 16:29:10.753073 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:10.752998 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" event={"ID":"b6363a48-8a18-40a5-a052-54a5cba03f8b","Type":"ContainerStarted","Data":"5fb38e1435f15ece9bbed786dc06d6b53137e1f90b846adb99396c02fab60852"} Jul 21 16:29:12.766714 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:12.766676 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" event={"ID":"b6363a48-8a18-40a5-a052-54a5cba03f8b","Type":"ContainerStarted","Data":"5d31189bf9813f59c382d0b27daee5a82b0914d0b3a7c586328b1c127045c59c"} Jul 21 16:29:12.790914 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:12.790847 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zqfn5" podStartSLOduration=1.521134344 podStartE2EDuration="3.790832221s" totalImagesPullingTime="2.269697877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:12.788525332 +0000 UTC m=+265.827361029" watchObservedRunningTime="2026-07-21 16:29:12.790832221 +0000 UTC m=+265.829667875" Jul 21 16:29:13.304587 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.304552 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wnhrd"] Jul 21 16:29:13.306862 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.306844 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.310606 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.310588 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 16:29:13.310680 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.310633 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 16:29:13.321933 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.321908 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wnhrd"] Jul 21 16:29:13.390120 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.390084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f4sz\" (UniqueName: \"kubernetes.io/projected/544aaefa-0957-4d81-a99e-8b22d71622de-kube-api-access-5f4sz\") pod \"cert-manager-webhook-6486ffd997-wnhrd\" (UID: \"544aaefa-0957-4d81-a99e-8b22d71622de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.390291 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.390151 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/544aaefa-0957-4d81-a99e-8b22d71622de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wnhrd\" (UID: \"544aaefa-0957-4d81-a99e-8b22d71622de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.490748 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.490717 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/544aaefa-0957-4d81-a99e-8b22d71622de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wnhrd\" (UID: \"544aaefa-0957-4d81-a99e-8b22d71622de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.490926 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.490769 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f4sz\" (UniqueName: \"kubernetes.io/projected/544aaefa-0957-4d81-a99e-8b22d71622de-kube-api-access-5f4sz\") pod \"cert-manager-webhook-6486ffd997-wnhrd\" (UID: \"544aaefa-0957-4d81-a99e-8b22d71622de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.511966 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.511934 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/544aaefa-0957-4d81-a99e-8b22d71622de-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wnhrd\" (UID: \"544aaefa-0957-4d81-a99e-8b22d71622de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.512687 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.512668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f4sz\" (UniqueName: \"kubernetes.io/projected/544aaefa-0957-4d81-a99e-8b22d71622de-kube-api-access-5f4sz\") pod \"cert-manager-webhook-6486ffd997-wnhrd\" (UID: \"544aaefa-0957-4d81-a99e-8b22d71622de\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.627998 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.627899 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:13.757580 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.757555 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wnhrd"] Jul 21 16:29:13.759562 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:29:13.759529 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod544aaefa_0957_4d81_a99e_8b22d71622de.slice/crio-d70a8dd369edbc077200ec948f3e71a28f799345c33a9d05090756027e7e4542 WatchSource:0}: Error finding container d70a8dd369edbc077200ec948f3e71a28f799345c33a9d05090756027e7e4542: Status 404 returned error can't find the container with id d70a8dd369edbc077200ec948f3e71a28f799345c33a9d05090756027e7e4542 Jul 21 16:29:13.771721 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:13.771694 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" event={"ID":"544aaefa-0957-4d81-a99e-8b22d71622de","Type":"ContainerStarted","Data":"d70a8dd369edbc077200ec948f3e71a28f799345c33a9d05090756027e7e4542"} Jul 21 16:29:16.787445 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:16.787404 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" event={"ID":"544aaefa-0957-4d81-a99e-8b22d71622de","Type":"ContainerStarted","Data":"942df25e61aa9ddbe24aa2851c233b08564110adc4e26a84de65f36d4dfbfb34"} Jul 21 16:29:16.787885 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:16.787518 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:16.803582 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:16.803493 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" podStartSLOduration=1.012251401 podStartE2EDuration="3.803478517s" totalImagesPullingTime="2.791227116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:16.802724242 +0000 UTC m=+269.841559931" watchObservedRunningTime="2026-07-21 16:29:16.803478517 +0000 UTC m=+269.842314179" Jul 21 16:29:19.895514 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:19.895481 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf"] Jul 21 16:29:19.897971 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:19.897949 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:19.900833 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:19.900784 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-pjx4l\"" Jul 21 16:29:19.906245 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:19.906221 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf"] Jul 21 16:29:19.944875 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:19.944847 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6239d1d-5c96-4427-bc34-318294f68cd6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gq9qf\" (UID: \"d6239d1d-5c96-4427-bc34-318294f68cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:19.945049 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:19.944887 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbn8d\" (UniqueName: \"kubernetes.io/projected/d6239d1d-5c96-4427-bc34-318294f68cd6-kube-api-access-mbn8d\") pod \"cert-manager-cainjector-559cdd8c87-gq9qf\" (UID: \"d6239d1d-5c96-4427-bc34-318294f68cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:20.046170 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.046131 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbn8d\" (UniqueName: \"kubernetes.io/projected/d6239d1d-5c96-4427-bc34-318294f68cd6-kube-api-access-mbn8d\") pod \"cert-manager-cainjector-559cdd8c87-gq9qf\" (UID: \"d6239d1d-5c96-4427-bc34-318294f68cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:20.046357 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.046240 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6239d1d-5c96-4427-bc34-318294f68cd6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gq9qf\" (UID: \"d6239d1d-5c96-4427-bc34-318294f68cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:20.058278 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.058255 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6239d1d-5c96-4427-bc34-318294f68cd6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gq9qf\" (UID: \"d6239d1d-5c96-4427-bc34-318294f68cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:20.058407 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.058386 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbn8d\" (UniqueName: \"kubernetes.io/projected/d6239d1d-5c96-4427-bc34-318294f68cd6-kube-api-access-mbn8d\") pod \"cert-manager-cainjector-559cdd8c87-gq9qf\" (UID: \"d6239d1d-5c96-4427-bc34-318294f68cd6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:20.208043 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.207952 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" Jul 21 16:29:20.335219 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.335193 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf"] Jul 21 16:29:20.337291 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:29:20.337261 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6239d1d_5c96_4427_bc34_318294f68cd6.slice/crio-392ac74aacbd04c789d1e2a95c46188fe4bc684dd2d991caf4c92f9a6fcb0c0b WatchSource:0}: Error finding container 392ac74aacbd04c789d1e2a95c46188fe4bc684dd2d991caf4c92f9a6fcb0c0b: Status 404 returned error can't find the container with id 392ac74aacbd04c789d1e2a95c46188fe4bc684dd2d991caf4c92f9a6fcb0c0b Jul 21 16:29:20.805808 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.805770 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" event={"ID":"d6239d1d-5c96-4427-bc34-318294f68cd6","Type":"ContainerStarted","Data":"115c2eccec2c3355bfacd16075404b730dde2c3ddd92e27381d23c8fe395a1cb"} Jul 21 16:29:20.805808 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.805830 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" event={"ID":"d6239d1d-5c96-4427-bc34-318294f68cd6","Type":"ContainerStarted","Data":"392ac74aacbd04c789d1e2a95c46188fe4bc684dd2d991caf4c92f9a6fcb0c0b"} Jul 21 16:29:20.822737 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:20.822689 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gq9qf" podStartSLOduration=1.822675497 podStartE2EDuration="1.822675497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:20.821230446 +0000 UTC m=+273.860066109" watchObservedRunningTime="2026-07-21 16:29:20.822675497 +0000 UTC m=+273.861511157" Jul 21 16:29:22.793460 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:22.793427 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wnhrd" Jul 21 16:29:47.352126 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:47.352097 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:29:47.356072 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:47.356047 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:29:47.364228 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:47.364206 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 16:29:51.115085 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.115051 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49"] Jul 21 16:29:51.117236 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.117221 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.120184 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.120148 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 16:29:51.120297 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.120182 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 16:29:51.120297 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.120278 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 16:29:51.120680 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.120663 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 16:29:51.120782 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.120692 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2dmv6\"" Jul 21 16:29:51.138068 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.138040 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49"] Jul 21 16:29:51.201146 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.201113 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4765e0fe-354c-44b6-9589-a56ca58799c4-manifests\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.201323 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.201151 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78hd7\" (UniqueName: \"kubernetes.io/projected/4765e0fe-354c-44b6-9589-a56ca58799c4-kube-api-access-78hd7\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.201323 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.201181 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4765e0fe-354c-44b6-9589-a56ca58799c4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.201323 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.201270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4765e0fe-354c-44b6-9589-a56ca58799c4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.201323 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.201307 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4765e0fe-354c-44b6-9589-a56ca58799c4-tmp\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.302656 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.302618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4765e0fe-354c-44b6-9589-a56ca58799c4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.302864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.302662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4765e0fe-354c-44b6-9589-a56ca58799c4-tmp\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.302864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.302788 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4765e0fe-354c-44b6-9589-a56ca58799c4-manifests\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.302864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.302854 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78hd7\" (UniqueName: \"kubernetes.io/projected/4765e0fe-354c-44b6-9589-a56ca58799c4-kube-api-access-78hd7\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.303014 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.302897 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4765e0fe-354c-44b6-9589-a56ca58799c4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.303179 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.303159 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/4765e0fe-354c-44b6-9589-a56ca58799c4-manifests\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.304974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.304946 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4765e0fe-354c-44b6-9589-a56ca58799c4-tmp\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.305273 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.305256 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4765e0fe-354c-44b6-9589-a56ca58799c4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.305331 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.305281 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4765e0fe-354c-44b6-9589-a56ca58799c4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.311774 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.311757 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78hd7\" (UniqueName: \"kubernetes.io/projected/4765e0fe-354c-44b6-9589-a56ca58799c4-kube-api-access-78hd7\") pod \"opendatahub-operator-controller-manager-7d658cff7b-frc49\" (UID: \"4765e0fe-354c-44b6-9589-a56ca58799c4\") " pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.428178 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.428089 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:51.569917 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.567985 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49"] Jul 21 16:29:51.573082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.573048 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 16:29:51.943542 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:51.943506 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" event={"ID":"4765e0fe-354c-44b6-9589-a56ca58799c4","Type":"ContainerStarted","Data":"94b0b2d05f3bc47fd30d763c448020768695a9487177f192b59ff0c71a62f632"} Jul 21 16:29:54.959747 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:54.959711 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" event={"ID":"4765e0fe-354c-44b6-9589-a56ca58799c4","Type":"ContainerStarted","Data":"88ec8d1f41a9ab17447913556dbee05c9d8e7004ec55148f5f6101e3ee3c87b1"} Jul 21 16:29:55.965376 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:55.965341 2557 generic.go:356] "Generic (PLEG): container finished" podID="4765e0fe-354c-44b6-9589-a56ca58799c4" containerID="88ec8d1f41a9ab17447913556dbee05c9d8e7004ec55148f5f6101e3ee3c87b1" exitCode=0 Jul 21 16:29:55.965745 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:55.965420 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" event={"ID":"4765e0fe-354c-44b6-9589-a56ca58799c4","Type":"ContainerDied","Data":"88ec8d1f41a9ab17447913556dbee05c9d8e7004ec55148f5f6101e3ee3c87b1"} Jul 21 16:29:56.972571 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:56.972483 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" event={"ID":"4765e0fe-354c-44b6-9589-a56ca58799c4","Type":"ContainerStarted","Data":"cb509bd99d09ee4f78e26b083793d98933da3ff03cc558c9493fe9095f334646"} Jul 21 16:29:56.973035 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:56.972613 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:29:56.999394 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:29:56.999347 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" podStartSLOduration=1.239126748 podStartE2EDuration="5.999332711s" totalImagesPullingTime="3.749409882s" totalInitContainerRuntime="1.010796081s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:29:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:29:56.996329554 +0000 UTC m=+310.035165217" watchObservedRunningTime="2026-07-21 16:29:56.999332711 +0000 UTC m=+310.038168373" Jul 21 16:30:07.979088 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:07.979056 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7d658cff7b-frc49" Jul 21 16:30:11.494842 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.494790 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jkxl5"] Jul 21 16:30:11.499292 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.499273 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.502177 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.502160 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 21 16:30:11.508566 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.508539 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jkxl5"] Jul 21 16:30:11.572545 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.572507 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7f26\" (UniqueName: \"kubernetes.io/projected/2b4adb52-eeb3-4c58-98d8-94be5ae928e6-kube-api-access-c7f26\") pod \"model-serving-api-54d89bd5fc-jkxl5\" (UID: \"2b4adb52-eeb3-4c58-98d8-94be5ae928e6\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.572545 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.572555 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b4adb52-eeb3-4c58-98d8-94be5ae928e6-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkxl5\" (UID: \"2b4adb52-eeb3-4c58-98d8-94be5ae928e6\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.673402 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.673366 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7f26\" (UniqueName: \"kubernetes.io/projected/2b4adb52-eeb3-4c58-98d8-94be5ae928e6-kube-api-access-c7f26\") pod \"model-serving-api-54d89bd5fc-jkxl5\" (UID: \"2b4adb52-eeb3-4c58-98d8-94be5ae928e6\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.673577 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.673419 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b4adb52-eeb3-4c58-98d8-94be5ae928e6-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkxl5\" (UID: \"2b4adb52-eeb3-4c58-98d8-94be5ae928e6\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.675973 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.675954 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b4adb52-eeb3-4c58-98d8-94be5ae928e6-tls-certs\") pod \"model-serving-api-54d89bd5fc-jkxl5\" (UID: \"2b4adb52-eeb3-4c58-98d8-94be5ae928e6\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.684901 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.684876 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7f26\" (UniqueName: \"kubernetes.io/projected/2b4adb52-eeb3-4c58-98d8-94be5ae928e6-kube-api-access-c7f26\") pod \"model-serving-api-54d89bd5fc-jkxl5\" (UID: \"2b4adb52-eeb3-4c58-98d8-94be5ae928e6\") " pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.811114 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.811026 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:11.986842 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:11.986797 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-jkxl5"] Jul 21 16:30:11.989121 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:30:11.989091 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b4adb52_eeb3_4c58_98d8_94be5ae928e6.slice/crio-c011e458c966c94b710073127b95a9b126c8f1983f6e71d80abb3a2685573b83 WatchSource:0}: Error finding container c011e458c966c94b710073127b95a9b126c8f1983f6e71d80abb3a2685573b83: Status 404 returned error can't find the container with id c011e458c966c94b710073127b95a9b126c8f1983f6e71d80abb3a2685573b83 Jul 21 16:30:12.042876 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:12.042840 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" event={"ID":"2b4adb52-eeb3-4c58-98d8-94be5ae928e6","Type":"ContainerStarted","Data":"c011e458c966c94b710073127b95a9b126c8f1983f6e71d80abb3a2685573b83"} Jul 21 16:30:15.058969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:15.058924 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" event={"ID":"2b4adb52-eeb3-4c58-98d8-94be5ae928e6","Type":"ContainerStarted","Data":"4809a7d21662063d1d3c3c0fd444f504bd53df059a1848828dad8031572c0398"} Jul 21 16:30:15.059364 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:15.059043 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:15.084011 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:15.083948 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" podStartSLOduration=1.526974106 podStartE2EDuration="4.083929033s" totalImagesPullingTime="2.556954927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:30:15.0830806 +0000 UTC m=+328.121916263" watchObservedRunningTime="2026-07-21 16:30:15.083929033 +0000 UTC m=+328.122764694" Jul 21 16:30:20.521704 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.521671 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6"] Jul 21 16:30:20.525240 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.525221 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:20.528256 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.528226 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 21 16:30:20.528375 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.528238 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jww4v\"" Jul 21 16:30:20.528857 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.528808 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g"] Jul 21 16:30:20.531893 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.531856 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:20.534080 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.533978 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6"] Jul 21 16:30:20.535134 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.535031 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 21 16:30:20.535304 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.535276 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ckn26\"" Jul 21 16:30:20.544285 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.544260 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g"] Jul 21 16:30:20.654770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.654738 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6g579\" (UniqueName: \"kubernetes.io/projected/492b1073-be50-405b-aef3-ae8c9407fc2f-kube-api-access-6g579\") pod \"kserve-controller-manager-d6f4b65b8-4kkm6\" (UID: \"492b1073-be50-405b-aef3-ae8c9407fc2f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:20.654966 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.654779 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qnd6g\" (UID: \"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:20.654966 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.654808 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/492b1073-be50-405b-aef3-ae8c9407fc2f-cert\") pod \"kserve-controller-manager-d6f4b65b8-4kkm6\" (UID: \"492b1073-be50-405b-aef3-ae8c9407fc2f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:20.654966 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.654889 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql5t5\" (UniqueName: \"kubernetes.io/projected/b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5-kube-api-access-ql5t5\") pod \"llmisvc-controller-manager-777c6bd7d5-qnd6g\" (UID: \"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:20.756136 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.756097 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6g579\" (UniqueName: \"kubernetes.io/projected/492b1073-be50-405b-aef3-ae8c9407fc2f-kube-api-access-6g579\") pod \"kserve-controller-manager-d6f4b65b8-4kkm6\" (UID: \"492b1073-be50-405b-aef3-ae8c9407fc2f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:20.756317 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.756148 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qnd6g\" (UID: \"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:20.756317 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.756177 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/492b1073-be50-405b-aef3-ae8c9407fc2f-cert\") pod \"kserve-controller-manager-d6f4b65b8-4kkm6\" (UID: \"492b1073-be50-405b-aef3-ae8c9407fc2f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:20.756317 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.756220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5t5\" (UniqueName: \"kubernetes.io/projected/b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5-kube-api-access-ql5t5\") pod \"llmisvc-controller-manager-777c6bd7d5-qnd6g\" (UID: \"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:20.756471 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:30:20.756309 2557 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 21 16:30:20.756471 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:30:20.756390 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492b1073-be50-405b-aef3-ae8c9407fc2f-cert podName:492b1073-be50-405b-aef3-ae8c9407fc2f nodeName:}" failed. No retries permitted until 2026-07-21 16:30:21.256370601 +0000 UTC m=+334.295206242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/492b1073-be50-405b-aef3-ae8c9407fc2f-cert") pod "kserve-controller-manager-d6f4b65b8-4kkm6" (UID: "492b1073-be50-405b-aef3-ae8c9407fc2f") : secret "kserve-webhook-server-cert" not found Jul 21 16:30:20.758688 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.758667 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-qnd6g\" (UID: \"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:20.773704 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.773646 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6g579\" (UniqueName: \"kubernetes.io/projected/492b1073-be50-405b-aef3-ae8c9407fc2f-kube-api-access-6g579\") pod \"kserve-controller-manager-d6f4b65b8-4kkm6\" (UID: \"492b1073-be50-405b-aef3-ae8c9407fc2f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:20.775047 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.775026 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql5t5\" (UniqueName: \"kubernetes.io/projected/b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5-kube-api-access-ql5t5\") pod \"llmisvc-controller-manager-777c6bd7d5-qnd6g\" (UID: \"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:20.847205 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:20.847167 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:21.025455 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.025428 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g"] Jul 21 16:30:21.027381 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:30:21.027359 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb4397e1d_6b9a_4f54_acce_9fa8b0dfdff5.slice/crio-1eb2980f30fdfad240b383078641ddb8823b5fab852a33a890ec91e9fa23ae9f WatchSource:0}: Error finding container 1eb2980f30fdfad240b383078641ddb8823b5fab852a33a890ec91e9fa23ae9f: Status 404 returned error can't find the container with id 1eb2980f30fdfad240b383078641ddb8823b5fab852a33a890ec91e9fa23ae9f Jul 21 16:30:21.087270 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.087235 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" event={"ID":"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5","Type":"ContainerStarted","Data":"1eb2980f30fdfad240b383078641ddb8823b5fab852a33a890ec91e9fa23ae9f"} Jul 21 16:30:21.116363 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.116327 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq"] Jul 21 16:30:21.120771 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.120753 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.124475 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.124454 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 21 16:30:21.124694 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.124671 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 21 16:30:21.124788 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.124680 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xh285\"" Jul 21 16:30:21.141644 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.141617 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq"] Jul 21 16:30:21.158699 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.158670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.158844 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.158743 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5998\" (UniqueName: \"kubernetes.io/projected/1a700735-0ab6-4fca-b7a3-cce620939005-kube-api-access-v5998\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.158844 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.158768 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a700735-0ab6-4fca-b7a3-cce620939005-tmp\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.259585 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.259552 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.259770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.259625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/492b1073-be50-405b-aef3-ae8c9407fc2f-cert\") pod \"kserve-controller-manager-d6f4b65b8-4kkm6\" (UID: \"492b1073-be50-405b-aef3-ae8c9407fc2f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:21.259770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.259664 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5998\" (UniqueName: \"kubernetes.io/projected/1a700735-0ab6-4fca-b7a3-cce620939005-kube-api-access-v5998\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.259770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.259688 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a700735-0ab6-4fca-b7a3-cce620939005-tmp\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.259770 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:30:21.259709 2557 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 21 16:30:21.259981 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:30:21.259781 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs podName:1a700735-0ab6-4fca-b7a3-cce620939005 nodeName:}" failed. No retries permitted until 2026-07-21 16:30:21.759763372 +0000 UTC m=+334.798599017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs") pod "kube-auth-proxy-6dcb5fbcb6-s62mq" (UID: "1a700735-0ab6-4fca-b7a3-cce620939005") : secret "kube-auth-proxy-tls" not found Jul 21 16:30:21.262158 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.262132 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a700735-0ab6-4fca-b7a3-cce620939005-tmp\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.262402 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.262381 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/492b1073-be50-405b-aef3-ae8c9407fc2f-cert\") pod \"kserve-controller-manager-d6f4b65b8-4kkm6\" (UID: \"492b1073-be50-405b-aef3-ae8c9407fc2f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:21.273673 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.273649 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5998\" (UniqueName: \"kubernetes.io/projected/1a700735-0ab6-4fca-b7a3-cce620939005-kube-api-access-v5998\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.437556 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.437477 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:21.573366 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.573343 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6"] Jul 21 16:30:21.575164 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:30:21.575128 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod492b1073_be50_405b_aef3_ae8c9407fc2f.slice/crio-85366b657c362af1521c4784022e56ac2b06572d80a2eefa14939a2cc5d0393c WatchSource:0}: Error finding container 85366b657c362af1521c4784022e56ac2b06572d80a2eefa14939a2cc5d0393c: Status 404 returned error can't find the container with id 85366b657c362af1521c4784022e56ac2b06572d80a2eefa14939a2cc5d0393c Jul 21 16:30:21.762690 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:21.762655 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:21.762904 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:30:21.762805 2557 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 21 16:30:21.762985 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:30:21.762904 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs podName:1a700735-0ab6-4fca-b7a3-cce620939005 nodeName:}" failed. No retries permitted until 2026-07-21 16:30:22.762885441 +0000 UTC m=+335.801721085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs") pod "kube-auth-proxy-6dcb5fbcb6-s62mq" (UID: "1a700735-0ab6-4fca-b7a3-cce620939005") : secret "kube-auth-proxy-tls" not found Jul 21 16:30:22.094189 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:22.094097 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" event={"ID":"492b1073-be50-405b-aef3-ae8c9407fc2f","Type":"ContainerStarted","Data":"85366b657c362af1521c4784022e56ac2b06572d80a2eefa14939a2cc5d0393c"} Jul 21 16:30:22.771580 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:22.771538 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:22.774382 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:22.774341 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a700735-0ab6-4fca-b7a3-cce620939005-tls-certs\") pod \"kube-auth-proxy-6dcb5fbcb6-s62mq\" (UID: \"1a700735-0ab6-4fca-b7a3-cce620939005\") " pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:22.931338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:22.931045 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" Jul 21 16:30:23.106021 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.105984 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq"] Jul 21 16:30:23.109080 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:30:23.109049 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a700735_0ab6_4fca_b7a3_cce620939005.slice/crio-c8bcabb395e45d3db3ea7385d0591115ff81dea3d5c4b327bad640f5ee676545 WatchSource:0}: Error finding container c8bcabb395e45d3db3ea7385d0591115ff81dea3d5c4b327bad640f5ee676545: Status 404 returned error can't find the container with id c8bcabb395e45d3db3ea7385d0591115ff81dea3d5c4b327bad640f5ee676545 Jul 21 16:30:23.207482 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.207440 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn"] Jul 21 16:30:23.212129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.212104 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.217674 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.217650 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 16:30:23.217936 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.217918 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 16:30:23.218048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.217941 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tzg5b\"" Jul 21 16:30:23.218107 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.218092 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 16:30:23.219738 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.219275 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 16:30:23.238574 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.238544 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn"] Jul 21 16:30:23.277537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.277501 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.277729 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.277542 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.277729 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.277575 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f813ed7c-954f-4567-9321-081bcee44a36-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.277729 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.277674 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f27wq\" (UniqueName: \"kubernetes.io/projected/f813ed7c-954f-4567-9321-081bcee44a36-kube-api-access-f27wq\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.277729 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.277716 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f813ed7c-954f-4567-9321-081bcee44a36-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.277934 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.277778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.277934 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.277812 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f813ed7c-954f-4567-9321-081bcee44a36-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.378987 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.378896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f813ed7c-954f-4567-9321-081bcee44a36-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.378987 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.378981 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.379188 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.379011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.379188 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.379070 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f813ed7c-954f-4567-9321-081bcee44a36-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.379188 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.379145 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f27wq\" (UniqueName: \"kubernetes.io/projected/f813ed7c-954f-4567-9321-081bcee44a36-kube-api-access-f27wq\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.379188 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.379177 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f813ed7c-954f-4567-9321-081bcee44a36-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.379392 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.379217 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.380047 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.380014 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f813ed7c-954f-4567-9321-081bcee44a36-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.381840 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.381793 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.381943 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.381879 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f813ed7c-954f-4567-9321-081bcee44a36-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.381943 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.381878 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.382038 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.381977 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f813ed7c-954f-4567-9321-081bcee44a36-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.389709 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.389681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f27wq\" (UniqueName: \"kubernetes.io/projected/f813ed7c-954f-4567-9321-081bcee44a36-kube-api-access-f27wq\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.390908 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.390882 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f813ed7c-954f-4567-9321-081bcee44a36-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-cf2fn\" (UID: \"f813ed7c-954f-4567-9321-081bcee44a36\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.522270 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.522238 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:23.718233 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:23.718206 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn"] Jul 21 16:30:24.107163 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:24.107117 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" event={"ID":"1a700735-0ab6-4fca-b7a3-cce620939005","Type":"ContainerStarted","Data":"c8bcabb395e45d3db3ea7385d0591115ff81dea3d5c4b327bad640f5ee676545"} Jul 21 16:30:24.292367 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:30:24.292331 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf813ed7c_954f_4567_9321_081bcee44a36.slice/crio-1cf463ab302e84fc3090d87ce73683d0255027229682d0d68316e37b4853abef WatchSource:0}: Error finding container 1cf463ab302e84fc3090d87ce73683d0255027229682d0d68316e37b4853abef: Status 404 returned error can't find the container with id 1cf463ab302e84fc3090d87ce73683d0255027229682d0d68316e37b4853abef Jul 21 16:30:25.116646 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:25.116604 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" event={"ID":"492b1073-be50-405b-aef3-ae8c9407fc2f","Type":"ContainerStarted","Data":"e43d6d760e98440c07a001d8c8bfc820067c7bfa3470116ddb09f26ab0d53304"} Jul 21 16:30:25.117211 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:25.117190 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:30:25.119474 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:25.119443 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" event={"ID":"f813ed7c-954f-4567-9321-081bcee44a36","Type":"ContainerStarted","Data":"1cf463ab302e84fc3090d87ce73683d0255027229682d0d68316e37b4853abef"} Jul 21 16:30:25.121476 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:25.121437 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" event={"ID":"b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5","Type":"ContainerStarted","Data":"50e9c6355c39017ae8b3f3ab8fefc5ee18f656860d7cec5f1fd3629fa03553a4"} Jul 21 16:30:25.121898 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:25.121878 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:25.147411 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:25.147073 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" podStartSLOduration=1.7888137290000001 podStartE2EDuration="5.147053181s" totalImagesPullingTime="3.358239452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:30:25.146083208 +0000 UTC m=+338.184918871" watchObservedRunningTime="2026-07-21 16:30:25.147053181 +0000 UTC m=+338.185888843" Jul 21 16:30:25.180411 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:25.180289 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" podStartSLOduration=1.277966953 podStartE2EDuration="5.180266852s" totalImagesPullingTime="3.902299899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:30:25.179188329 +0000 UTC m=+338.218023993" watchObservedRunningTime="2026-07-21 16:30:25.180266852 +0000 UTC m=+338.219102514" Jul 21 16:30:26.069494 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:26.069463 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-jkxl5" Jul 21 16:30:27.190676 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:27.190635 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:30:27.191074 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:27.190712 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:30:28.139995 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:28.139955 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" event={"ID":"1a700735-0ab6-4fca-b7a3-cce620939005","Type":"ContainerStarted","Data":"5ba20f464620f2133f44f1a79e98a4099f30ed95ef6f892716e3d0c3560efd61"} Jul 21 16:30:28.141539 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:28.141513 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" event={"ID":"f813ed7c-954f-4567-9321-081bcee44a36","Type":"ContainerStarted","Data":"9e9b7f76c27fba06be67098b016b15f944d20c883f0a2c10e6a09ed77058979d"} Jul 21 16:30:28.141687 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:28.141671 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:28.143249 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:28.143215 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-cf2fn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 16:30:28.143324 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:28.143267 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" podUID="f813ed7c-954f-4567-9321-081bcee44a36" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 16:30:28.175933 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:28.175882 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6dcb5fbcb6-s62mq" podStartSLOduration=3.149775191 podStartE2EDuration="7.175865436s" totalImagesPullingTime="4.026090245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:30:28.174283187 +0000 UTC m=+341.213118851" watchObservedRunningTime="2026-07-21 16:30:28.175865436 +0000 UTC m=+341.214701098" Jul 21 16:30:28.223530 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:28.223454 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" podStartSLOduration=2.327801766 podStartE2EDuration="5.223431612s" totalImagesPullingTime="2.895629846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:30:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:30:28.221554938 +0000 UTC m=+341.260390601" watchObservedRunningTime="2026-07-21 16:30:28.223431612 +0000 UTC m=+341.262267275" Jul 21 16:30:29.146607 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:29.146581 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-cf2fn" Jul 21 16:30:56.129999 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:56.129917 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-qnd6g" Jul 21 16:30:57.139060 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:30:57.139027 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4kkm6" Jul 21 16:31:47.725287 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.725255 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d"] Jul 21 16:31:47.727760 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.727744 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" Jul 21 16:31:47.730979 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.730957 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 21 16:31:47.731139 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.731000 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 21 16:31:47.732444 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.732424 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sltn6\"" Jul 21 16:31:47.732573 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.732490 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 21 16:31:47.743130 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.743107 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d"] Jul 21 16:31:47.819595 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.819568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w45s\" (UniqueName: \"kubernetes.io/projected/bd983f9c-67c5-40d2-9176-7d0188e8446d-kube-api-access-7w45s\") pod \"dns-operator-controller-manager-864dd78c4c-lrb7d\" (UID: \"bd983f9c-67c5-40d2-9176-7d0188e8446d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" Jul 21 16:31:47.919997 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.919968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w45s\" (UniqueName: \"kubernetes.io/projected/bd983f9c-67c5-40d2-9176-7d0188e8446d-kube-api-access-7w45s\") pod \"dns-operator-controller-manager-864dd78c4c-lrb7d\" (UID: \"bd983f9c-67c5-40d2-9176-7d0188e8446d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" Jul 21 16:31:47.929752 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:47.929721 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w45s\" (UniqueName: \"kubernetes.io/projected/bd983f9c-67c5-40d2-9176-7d0188e8446d-kube-api-access-7w45s\") pod \"dns-operator-controller-manager-864dd78c4c-lrb7d\" (UID: \"bd983f9c-67c5-40d2-9176-7d0188e8446d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" Jul 21 16:31:48.038606 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:48.038522 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" Jul 21 16:31:48.190945 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:48.190923 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d"] Jul 21 16:31:48.193446 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:31:48.193416 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd983f9c_67c5_40d2_9176_7d0188e8446d.slice/crio-fdfe1227015338e5e385491c43f2c3ff28fc21252b9bcc374e8d153e55035195 WatchSource:0}: Error finding container fdfe1227015338e5e385491c43f2c3ff28fc21252b9bcc374e8d153e55035195: Status 404 returned error can't find the container with id fdfe1227015338e5e385491c43f2c3ff28fc21252b9bcc374e8d153e55035195 Jul 21 16:31:48.537164 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:48.537130 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" event={"ID":"bd983f9c-67c5-40d2-9176-7d0188e8446d","Type":"ContainerStarted","Data":"fdfe1227015338e5e385491c43f2c3ff28fc21252b9bcc374e8d153e55035195"} Jul 21 16:31:51.072314 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.072279 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz"] Jul 21 16:31:51.074697 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.074675 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:31:51.077686 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.077665 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-sw2tl\"" Jul 21 16:31:51.086479 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.086460 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz"] Jul 21 16:31:51.147363 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.147335 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96mr4\" (UniqueName: \"kubernetes.io/projected/a51bf04e-a872-4ed4-84c7-38a4894036ea-kube-api-access-96mr4\") pod \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" (UID: \"a51bf04e-a872-4ed4-84c7-38a4894036ea\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:31:51.248234 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.248200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96mr4\" (UniqueName: \"kubernetes.io/projected/a51bf04e-a872-4ed4-84c7-38a4894036ea-kube-api-access-96mr4\") pod \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" (UID: \"a51bf04e-a872-4ed4-84c7-38a4894036ea\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:31:51.263343 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.263305 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96mr4\" (UniqueName: \"kubernetes.io/projected/a51bf04e-a872-4ed4-84c7-38a4894036ea-kube-api-access-96mr4\") pod \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" (UID: \"a51bf04e-a872-4ed4-84c7-38a4894036ea\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:31:51.386198 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.386112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:31:51.525866 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.525839 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz"] Jul 21 16:31:51.527578 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:31:51.527540 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51bf04e_a872_4ed4_84c7_38a4894036ea.slice/crio-214c33f34b59059d38aefbe01958e57f902869b2a54a612dd5a8420742ef9f31 WatchSource:0}: Error finding container 214c33f34b59059d38aefbe01958e57f902869b2a54a612dd5a8420742ef9f31: Status 404 returned error can't find the container with id 214c33f34b59059d38aefbe01958e57f902869b2a54a612dd5a8420742ef9f31 Jul 21 16:31:51.553834 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.553782 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" event={"ID":"bd983f9c-67c5-40d2-9176-7d0188e8446d","Type":"ContainerStarted","Data":"974ba4645b4c6e03bb70984e90af3df86d2f1695a693d2a5c0f2e0019853bfe6"} Jul 21 16:31:51.553937 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.553922 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" Jul 21 16:31:51.555066 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.555038 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" event={"ID":"a51bf04e-a872-4ed4-84c7-38a4894036ea","Type":"ContainerStarted","Data":"214c33f34b59059d38aefbe01958e57f902869b2a54a612dd5a8420742ef9f31"} Jul 21 16:31:51.604243 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:51.604202 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" podStartSLOduration=2.263884768 podStartE2EDuration="4.604187773s" totalImagesPullingTime="2.340303005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:31:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:31:51.599525462 +0000 UTC m=+424.638361124" watchObservedRunningTime="2026-07-21 16:31:51.604187773 +0000 UTC m=+424.643023436" Jul 21 16:31:54.572437 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:54.572403 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" event={"ID":"a51bf04e-a872-4ed4-84c7-38a4894036ea","Type":"ContainerStarted","Data":"e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464"} Jul 21 16:31:54.572811 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:54.572549 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:31:54.607741 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:31:54.607695 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" podStartSLOduration=0.760524849 podStartE2EDuration="3.607681301s" totalImagesPullingTime="2.847156452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:31:54.604637675 +0000 UTC m=+427.643473338" watchObservedRunningTime="2026-07-21 16:31:54.607681301 +0000 UTC m=+427.646516962" Jul 21 16:32:02.564148 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:02.564118 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-lrb7d" Jul 21 16:32:05.579360 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:05.579330 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:15.359244 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.359165 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr"] Jul 21 16:32:15.362006 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.361985 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.364755 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.364735 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qcfkb\"" Jul 21 16:32:15.378525 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.378505 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr"] Jul 21 16:32:15.462547 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.462510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjggm\" (UniqueName: \"kubernetes.io/projected/51de4f69-ee30-4cc7-8256-d801743dfba5-kube-api-access-xjggm\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.462729 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.462582 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51de4f69-ee30-4cc7-8256-d801743dfba5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.563647 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.563607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51de4f69-ee30-4cc7-8256-d801743dfba5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.563889 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.563676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjggm\" (UniqueName: \"kubernetes.io/projected/51de4f69-ee30-4cc7-8256-d801743dfba5-kube-api-access-xjggm\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.564097 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.564077 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51de4f69-ee30-4cc7-8256-d801743dfba5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.572442 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.572419 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjggm\" (UniqueName: \"kubernetes.io/projected/51de4f69-ee30-4cc7-8256-d801743dfba5-kube-api-access-xjggm\") pod \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.643964 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.643882 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr"] Jul 21 16:32:15.644175 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.644164 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:15.652243 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.652212 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr"] Jul 21 16:32:15.673359 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.673328 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d"] Jul 21 16:32:15.676276 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.676251 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:15.695647 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.695613 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d"] Jul 21 16:32:15.699597 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.699567 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz"] Jul 21 16:32:15.699849 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.699797 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" containerName="manager" containerID="cri-o://e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464" gracePeriod=2 Jul 21 16:32:15.710374 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.710343 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz"] Jul 21 16:32:15.731158 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:15.731116 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:15.735243 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.735219 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck"] Jul 21 16:32:15.735812 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.735790 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" containerName="manager" Jul 21 16:32:15.735928 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.735816 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" containerName="manager" Jul 21 16:32:15.735995 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.735947 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" containerName="manager" Jul 21 16:32:15.738627 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.738610 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" Jul 21 16:32:15.752108 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.752083 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck"] Jul 21 16:32:15.765292 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:15.765257 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:15.865658 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.865623 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0047dc72-1bce-496a-9fba-fff0ee919b41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-spn9d\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:15.865844 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.865684 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2tc4\" (UniqueName: \"kubernetes.io/projected/5f2e30d1-f72e-4771-8639-9d6970d378ff-kube-api-access-z2tc4\") pod \"limitador-operator-controller-manager-859d7b78bd-4w9ck\" (UID: \"5f2e30d1-f72e-4771-8639-9d6970d378ff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" Jul 21 16:32:15.865844 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.865729 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkzjb\" (UniqueName: \"kubernetes.io/projected/0047dc72-1bce-496a-9fba-fff0ee919b41-kube-api-access-xkzjb\") pod \"kuadrant-operator-controller-manager-55467d9f7-spn9d\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:15.934494 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.934466 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:15.937065 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:15.937036 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:15.966499 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.966469 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0047dc72-1bce-496a-9fba-fff0ee919b41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-spn9d\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:15.966635 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.966516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2tc4\" (UniqueName: \"kubernetes.io/projected/5f2e30d1-f72e-4771-8639-9d6970d378ff-kube-api-access-z2tc4\") pod \"limitador-operator-controller-manager-859d7b78bd-4w9ck\" (UID: \"5f2e30d1-f72e-4771-8639-9d6970d378ff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" Jul 21 16:32:15.966696 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.966647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkzjb\" (UniqueName: \"kubernetes.io/projected/0047dc72-1bce-496a-9fba-fff0ee919b41-kube-api-access-xkzjb\") pod \"kuadrant-operator-controller-manager-55467d9f7-spn9d\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:15.966917 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.966899 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0047dc72-1bce-496a-9fba-fff0ee919b41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-spn9d\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:15.984309 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.984281 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkzjb\" (UniqueName: \"kubernetes.io/projected/0047dc72-1bce-496a-9fba-fff0ee919b41-kube-api-access-xkzjb\") pod \"kuadrant-operator-controller-manager-55467d9f7-spn9d\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:15.984410 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.984311 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2tc4\" (UniqueName: \"kubernetes.io/projected/5f2e30d1-f72e-4771-8639-9d6970d378ff-kube-api-access-z2tc4\") pod \"limitador-operator-controller-manager-859d7b78bd-4w9ck\" (UID: \"5f2e30d1-f72e-4771-8639-9d6970d378ff\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" Jul 21 16:32:15.989117 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:15.989100 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:16.067081 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.067050 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96mr4\" (UniqueName: \"kubernetes.io/projected/a51bf04e-a872-4ed4-84c7-38a4894036ea-kube-api-access-96mr4\") pod \"a51bf04e-a872-4ed4-84c7-38a4894036ea\" (UID: \"a51bf04e-a872-4ed4-84c7-38a4894036ea\") " Jul 21 16:32:16.069259 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.069231 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51bf04e-a872-4ed4-84c7-38a4894036ea-kube-api-access-96mr4" (OuterVolumeSpecName: "kube-api-access-96mr4") pod "a51bf04e-a872-4ed4-84c7-38a4894036ea" (UID: "a51bf04e-a872-4ed4-84c7-38a4894036ea"). InnerVolumeSpecName "kube-api-access-96mr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:32:16.075170 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.075140 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" Jul 21 16:32:16.136213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.136182 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d"] Jul 21 16:32:16.139736 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:32:16.139704 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0047dc72_1bce_496a_9fba_fff0ee919b41.slice/crio-6c1be463e1d7bb443a5ba661b03643c864da311af8ea0a7113296583b15e2a92 WatchSource:0}: Error finding container 6c1be463e1d7bb443a5ba661b03643c864da311af8ea0a7113296583b15e2a92: Status 404 returned error can't find the container with id 6c1be463e1d7bb443a5ba661b03643c864da311af8ea0a7113296583b15e2a92 Jul 21 16:32:16.168373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.168345 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96mr4\" (UniqueName: \"kubernetes.io/projected/a51bf04e-a872-4ed4-84c7-38a4894036ea-kube-api-access-96mr4\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:32:16.227663 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.227637 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck"] Jul 21 16:32:16.229755 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:32:16.229731 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2e30d1_f72e_4771_8639_9d6970d378ff.slice/crio-6a11d037c9614eb6d963b0b1250705fcefefcbde2710b37231c530b2c4e812d6 WatchSource:0}: Error finding container 6a11d037c9614eb6d963b0b1250705fcefefcbde2710b37231c530b2c4e812d6: Status 404 returned error can't find the container with id 6a11d037c9614eb6d963b0b1250705fcefefcbde2710b37231c530b2c4e812d6 Jul 21 16:32:16.692022 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.691986 2557 generic.go:356] "Generic (PLEG): container finished" podID="a51bf04e-a872-4ed4-84c7-38a4894036ea" containerID="e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464" exitCode=0 Jul 21 16:32:16.692523 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.692033 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:16.692523 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.692107 2557 scope.go:117] "RemoveContainer" containerID="e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464" Jul 21 16:32:16.694463 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.694430 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" event={"ID":"5f2e30d1-f72e-4771-8639-9d6970d378ff","Type":"ContainerStarted","Data":"7d987236f4548ab8b5b0fa89b2f0e4c377725d40a7036c85a2b8df9e00d9c68c"} Jul 21 16:32:16.694567 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.694485 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" event={"ID":"5f2e30d1-f72e-4771-8639-9d6970d378ff","Type":"ContainerStarted","Data":"6a11d037c9614eb6d963b0b1250705fcefefcbde2710b37231c530b2c4e812d6"} Jul 21 16:32:16.694790 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.694751 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" Jul 21 16:32:16.695601 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:16.695534 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:16.696549 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.696497 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" event={"ID":"0047dc72-1bce-496a-9fba-fff0ee919b41","Type":"ContainerStarted","Data":"6c1be463e1d7bb443a5ba661b03643c864da311af8ea0a7113296583b15e2a92"} Jul 21 16:32:16.698492 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:16.698276 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:16.728271 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.728246 2557 scope.go:117] "RemoveContainer" containerID="e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464" Jul 21 16:32:16.728588 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:16.728565 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464\": container with ID starting with e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464 not found: ID does not exist" containerID="e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464" Jul 21 16:32:16.728692 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.728597 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464"} err="failed to get container status \"e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464\": rpc error: code = NotFound desc = could not find container \"e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464\": container with ID starting with e698d8387e487d784403ac9c60e3f3051ed000c1cdc8644435224f55ec9fd464 not found: ID does not exist" Jul 21 16:32:16.731815 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:16.731767 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" podStartSLOduration=1.731752757 podStartE2EDuration="1.731752757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:32:16.7297436 +0000 UTC m=+449.768579253" watchObservedRunningTime="2026-07-21 16:32:16.731752757 +0000 UTC m=+449.770588418" Jul 21 16:32:16.732471 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:16.732434 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:17.489073 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:17.489023 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rrvbz\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rrvbz" Jul 21 16:32:17.491578 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:17.491546 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a51bf04e-a872-4ed4-84c7-38a4894036ea" path="/var/lib/kubelet/pods/a51bf04e-a872-4ed4-84c7-38a4894036ea/volumes" Jul 21 16:32:18.187071 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:18.187023 2557 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-k4ddr_kuadrant-system_51de4f69-ee30-4cc7-8256-d801743dfba5_0(cb6b7c7124f6a7df0582422eb6f4a0239444ae6ce732319a67996e83de9387a4): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-k4ddr to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 21 16:32:18.187424 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:18.187097 2557 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-k4ddr_kuadrant-system_51de4f69-ee30-4cc7-8256-d801743dfba5_0(cb6b7c7124f6a7df0582422eb6f4a0239444ae6ce732319a67996e83de9387a4): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-k4ddr to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:18.709176 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.709147 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:18.716392 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.716372 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:18.719033 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:18.719009 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="51de4f69-ee30-4cc7-8256-d801743dfba5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:18.790178 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.790150 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51de4f69-ee30-4cc7-8256-d801743dfba5-extensions-socket-volume\") pod \"51de4f69-ee30-4cc7-8256-d801743dfba5\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " Jul 21 16:32:18.790304 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.790191 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjggm\" (UniqueName: \"kubernetes.io/projected/51de4f69-ee30-4cc7-8256-d801743dfba5-kube-api-access-xjggm\") pod \"51de4f69-ee30-4cc7-8256-d801743dfba5\" (UID: \"51de4f69-ee30-4cc7-8256-d801743dfba5\") " Jul 21 16:32:18.790404 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.790381 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51de4f69-ee30-4cc7-8256-d801743dfba5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "51de4f69-ee30-4cc7-8256-d801743dfba5" (UID: "51de4f69-ee30-4cc7-8256-d801743dfba5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:32:18.792231 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.792210 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51de4f69-ee30-4cc7-8256-d801743dfba5-kube-api-access-xjggm" (OuterVolumeSpecName: "kube-api-access-xjggm") pod "51de4f69-ee30-4cc7-8256-d801743dfba5" (UID: "51de4f69-ee30-4cc7-8256-d801743dfba5"). InnerVolumeSpecName "kube-api-access-xjggm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:32:18.890753 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.890722 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51de4f69-ee30-4cc7-8256-d801743dfba5-extensions-socket-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:32:18.890753 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:18.890748 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjggm\" (UniqueName: \"kubernetes.io/projected/51de4f69-ee30-4cc7-8256-d801743dfba5-kube-api-access-xjggm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:32:19.491015 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:19.490980 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51de4f69-ee30-4cc7-8256-d801743dfba5" path="/var/lib/kubelet/pods/51de4f69-ee30-4cc7-8256-d801743dfba5/volumes" Jul 21 16:32:19.715874 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:19.715816 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:19.721307 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:19.721272 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="51de4f69-ee30-4cc7-8256-d801743dfba5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:20.723242 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:20.723207 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" event={"ID":"0047dc72-1bce-496a-9fba-fff0ee919b41","Type":"ContainerStarted","Data":"94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5"} Jul 21 16:32:20.723614 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:20.723323 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:20.725971 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:20.725947 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-k4ddr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="51de4f69-ee30-4cc7-8256-d801743dfba5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-k4ddr" Jul 21 16:32:20.750379 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:20.750333 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" podStartSLOduration=1.684435046 podStartE2EDuration="5.750313661s" totalImagesPullingTime="4.065878615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:32:20.747096342 +0000 UTC m=+453.785932004" watchObservedRunningTime="2026-07-21 16:32:20.750313661 +0000 UTC m=+453.789149324" Jul 21 16:32:27.705554 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:27.705471 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4w9ck" Jul 21 16:32:31.730711 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:31.730677 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:31.997864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:31.997761 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m"] Jul 21 16:32:32.002472 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.002454 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.021405 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.021377 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m"] Jul 21 16:32:32.082810 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.082769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97j5z\" (UniqueName: \"kubernetes.io/projected/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-kube-api-access-97j5z\") pod \"kuadrant-operator-controller-manager-86878c769f-xks6m\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.082982 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.082858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xks6m\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.183422 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.183382 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97j5z\" (UniqueName: \"kubernetes.io/projected/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-kube-api-access-97j5z\") pod \"kuadrant-operator-controller-manager-86878c769f-xks6m\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.183617 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.183469 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xks6m\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.183811 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.183794 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xks6m\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.194960 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.194931 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97j5z\" (UniqueName: \"kubernetes.io/projected/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-kube-api-access-97j5z\") pod \"kuadrant-operator-controller-manager-86878c769f-xks6m\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.313481 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.313379 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.446656 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.446631 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m"] Jul 21 16:32:32.447749 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:32:32.447723 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode867ca42_c496_4dfe_8ab4_9f0bdf9cfb0c.slice/crio-d74cde9efd1b792d83ce7142e1a8337f1813c2046f48ed4bd4c8ab01be03c41f WatchSource:0}: Error finding container d74cde9efd1b792d83ce7142e1a8337f1813c2046f48ed4bd4c8ab01be03c41f: Status 404 returned error can't find the container with id d74cde9efd1b792d83ce7142e1a8337f1813c2046f48ed4bd4c8ab01be03c41f Jul 21 16:32:32.786968 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.786929 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" event={"ID":"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c","Type":"ContainerStarted","Data":"276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1"} Jul 21 16:32:32.787555 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.786984 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" event={"ID":"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c","Type":"ContainerStarted","Data":"d74cde9efd1b792d83ce7142e1a8337f1813c2046f48ed4bd4c8ab01be03c41f"} Jul 21 16:32:32.787555 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.787077 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:32.845060 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:32.845000 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" podStartSLOduration=1.844980038 podStartE2EDuration="1.844980038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:32:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:32:32.844015732 +0000 UTC m=+465.882851396" watchObservedRunningTime="2026-07-21 16:32:32.844980038 +0000 UTC m=+465.883815701" Jul 21 16:32:43.793870 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:43.793808 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:32:43.852445 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:43.852413 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d"] Jul 21 16:32:43.852683 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:43.852651 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" podUID="0047dc72-1bce-496a-9fba-fff0ee919b41" containerName="manager" containerID="cri-o://94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5" gracePeriod=10 Jul 21 16:32:44.111181 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.111157 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:44.188474 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.188438 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkzjb\" (UniqueName: \"kubernetes.io/projected/0047dc72-1bce-496a-9fba-fff0ee919b41-kube-api-access-xkzjb\") pod \"0047dc72-1bce-496a-9fba-fff0ee919b41\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " Jul 21 16:32:44.188639 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.188489 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0047dc72-1bce-496a-9fba-fff0ee919b41-extensions-socket-volume\") pod \"0047dc72-1bce-496a-9fba-fff0ee919b41\" (UID: \"0047dc72-1bce-496a-9fba-fff0ee919b41\") " Jul 21 16:32:44.189047 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.189022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0047dc72-1bce-496a-9fba-fff0ee919b41-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0047dc72-1bce-496a-9fba-fff0ee919b41" (UID: "0047dc72-1bce-496a-9fba-fff0ee919b41"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:32:44.190552 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.190529 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0047dc72-1bce-496a-9fba-fff0ee919b41-kube-api-access-xkzjb" (OuterVolumeSpecName: "kube-api-access-xkzjb") pod "0047dc72-1bce-496a-9fba-fff0ee919b41" (UID: "0047dc72-1bce-496a-9fba-fff0ee919b41"). InnerVolumeSpecName "kube-api-access-xkzjb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:32:44.289325 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.289287 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkzjb\" (UniqueName: \"kubernetes.io/projected/0047dc72-1bce-496a-9fba-fff0ee919b41-kube-api-access-xkzjb\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:32:44.289325 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.289320 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0047dc72-1bce-496a-9fba-fff0ee919b41-extensions-socket-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:32:44.848053 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.848021 2557 generic.go:356] "Generic (PLEG): container finished" podID="0047dc72-1bce-496a-9fba-fff0ee919b41" containerID="94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5" exitCode=0 Jul 21 16:32:44.848541 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.848061 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" Jul 21 16:32:44.848541 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.848109 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" event={"ID":"0047dc72-1bce-496a-9fba-fff0ee919b41","Type":"ContainerDied","Data":"94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5"} Jul 21 16:32:44.848541 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.848169 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d" event={"ID":"0047dc72-1bce-496a-9fba-fff0ee919b41","Type":"ContainerDied","Data":"6c1be463e1d7bb443a5ba661b03643c864da311af8ea0a7113296583b15e2a92"} Jul 21 16:32:44.848541 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.848204 2557 scope.go:117] "RemoveContainer" containerID="94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5" Jul 21 16:32:44.857786 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.857573 2557 scope.go:117] "RemoveContainer" containerID="94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5" Jul 21 16:32:44.857873 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:32:44.857855 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5\": container with ID starting with 94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5 not found: ID does not exist" containerID="94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5" Jul 21 16:32:44.857919 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.857882 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5"} err="failed to get container status \"94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5\": rpc error: code = NotFound desc = could not find container \"94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5\": container with ID starting with 94dbbfb069c0ea05ae3dad24d5b51bc2effc1b2cace8513e2cbcc85a61e4bbd5 not found: ID does not exist" Jul 21 16:32:44.871541 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.871514 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d"] Jul 21 16:32:44.876300 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:44.876279 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-spn9d"] Jul 21 16:32:45.489483 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:32:45.489451 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0047dc72-1bce-496a-9fba-fff0ee919b41" path="/var/lib/kubelet/pods/0047dc72-1bce-496a-9fba-fff0ee919b41/volumes" Jul 21 16:34:08.382338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.382257 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ckvzg"] Jul 21 16:34:08.382847 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.382677 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0047dc72-1bce-496a-9fba-fff0ee919b41" containerName="manager" Jul 21 16:34:08.382847 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.382694 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0047dc72-1bce-496a-9fba-fff0ee919b41" containerName="manager" Jul 21 16:34:08.382847 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.382802 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0047dc72-1bce-496a-9fba-fff0ee919b41" containerName="manager" Jul 21 16:34:08.385223 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.385203 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.389159 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.389140 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bjkxz\"" Jul 21 16:34:08.389254 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.389141 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 21 16:34:08.406270 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.406239 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ckvzg"] Jul 21 16:34:08.519630 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.519596 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ckvzg\" (UID: \"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef\") " pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.519796 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.519664 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24qll\" (UniqueName: \"kubernetes.io/projected/e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef-kube-api-access-24qll\") pod \"payload-processing-5bf47cd988-ckvzg\" (UID: \"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef\") " pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.621030 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.620999 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ckvzg\" (UID: \"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef\") " pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.621216 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.621065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24qll\" (UniqueName: \"kubernetes.io/projected/e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef-kube-api-access-24qll\") pod \"payload-processing-5bf47cd988-ckvzg\" (UID: \"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef\") " pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.621714 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.621694 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef-plugins-config-volume\") pod \"payload-processing-5bf47cd988-ckvzg\" (UID: \"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef\") " pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.631134 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.631113 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24qll\" (UniqueName: \"kubernetes.io/projected/e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef-kube-api-access-24qll\") pod \"payload-processing-5bf47cd988-ckvzg\" (UID: \"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef\") " pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.696795 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.696707 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:08.836546 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:08.836512 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-ckvzg"] Jul 21 16:34:08.838081 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:34:08.838047 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9db0f50_ae6b_4eab_bc2a_1238b9b5b1ef.slice/crio-82274a25de52155be0a274d7a14df989819c05fb9ba614bdd314fad287b414a8 WatchSource:0}: Error finding container 82274a25de52155be0a274d7a14df989819c05fb9ba614bdd314fad287b414a8: Status 404 returned error can't find the container with id 82274a25de52155be0a274d7a14df989819c05fb9ba614bdd314fad287b414a8 Jul 21 16:34:09.256378 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:09.256346 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" event={"ID":"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef","Type":"ContainerStarted","Data":"82274a25de52155be0a274d7a14df989819c05fb9ba614bdd314fad287b414a8"} Jul 21 16:34:11.269297 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:11.269262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" event={"ID":"e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef","Type":"ContainerStarted","Data":"0fbebcbe932d692ca7b0ee6cb0783d629c92b3e28771292ce95d4cfe1109e415"} Jul 21 16:34:11.269764 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:11.269347 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:11.287512 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:11.287463 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" podStartSLOduration=1.5280379320000002 podStartE2EDuration="3.287445922s" totalImagesPullingTime="1.75940799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:34:11.286074934 +0000 UTC m=+564.324910596" watchObservedRunningTime="2026-07-21 16:34:11.287445922 +0000 UTC m=+564.326281586" Jul 21 16:34:22.276707 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:22.276676 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-ckvzg" Jul 21 16:34:45.685427 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.685392 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq"] Jul 21 16:34:45.688149 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.688131 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.691155 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.691129 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 16:34:45.691275 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.691183 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kk782\"" Jul 21 16:34:45.692410 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.692395 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 16:34:45.692573 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.692557 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 21 16:34:45.699786 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.699767 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq"] Jul 21 16:34:45.743373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.743337 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r76s\" (UniqueName: \"kubernetes.io/projected/570120ea-b8c6-4e44-ab85-e77360155287-kube-api-access-9r76s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.743530 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.743437 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.743530 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.743508 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.743659 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.743535 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570120ea-b8c6-4e44-ab85-e77360155287-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.743659 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.743593 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.743765 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.743666 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.844937 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.844901 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845124 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.844943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570120ea-b8c6-4e44-ab85-e77360155287-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845124 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.844978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845124 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.845033 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845279 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.845127 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r76s\" (UniqueName: \"kubernetes.io/projected/570120ea-b8c6-4e44-ab85-e77360155287-kube-api-access-9r76s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845279 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.845181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845381 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.845349 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845443 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.845401 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.845541 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.845512 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.847342 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.847319 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/570120ea-b8c6-4e44-ab85-e77360155287-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.847584 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.847565 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/570120ea-b8c6-4e44-ab85-e77360155287-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.853676 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.853655 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r76s\" (UniqueName: \"kubernetes.io/projected/570120ea-b8c6-4e44-ab85-e77360155287-kube-api-access-9r76s\") pod \"e2e-distinct-simulated-kserve-59975d69f9-x2ftq\" (UID: \"570120ea-b8c6-4e44-ab85-e77360155287\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:45.998012 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:45.997979 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:46.134830 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:46.134791 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq"] Jul 21 16:34:46.139222 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:34:46.139190 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570120ea_b8c6_4e44_ab85_e77360155287.slice/crio-bef5a5de939a16d714908a2ff19e18c5fd8f3b9c742525972548cdc942742a01 WatchSource:0}: Error finding container bef5a5de939a16d714908a2ff19e18c5fd8f3b9c742525972548cdc942742a01: Status 404 returned error can't find the container with id bef5a5de939a16d714908a2ff19e18c5fd8f3b9c742525972548cdc942742a01 Jul 21 16:34:46.445885 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:46.445764 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" event={"ID":"570120ea-b8c6-4e44-ab85-e77360155287","Type":"ContainerStarted","Data":"bef5a5de939a16d714908a2ff19e18c5fd8f3b9c742525972548cdc942742a01"} Jul 21 16:34:47.385508 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:47.385468 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:34:47.387834 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:47.387796 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:34:47.453100 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:47.453069 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" event={"ID":"570120ea-b8c6-4e44-ab85-e77360155287","Type":"ContainerStarted","Data":"1b7e6bc571d10b4c81be780ac4a21e9a9229a5f7e96ed7e383caf0244f20eda6"} Jul 21 16:34:47.473260 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:47.473182 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" podStartSLOduration=1.39436308 podStartE2EDuration="2.473168656s" totalImagesPullingTime="1.078805576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:34:47.470697557 +0000 UTC m=+600.509533222" watchObservedRunningTime="2026-07-21 16:34:47.473168656 +0000 UTC m=+600.512004318" Jul 21 16:34:55.998363 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:55.998330 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:56.000204 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:55.998402 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:56.010771 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:56.010748 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:34:56.514684 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:34:56.514656 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-x2ftq" Jul 21 16:35:03.563614 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.563580 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8"] Jul 21 16:35:03.566223 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.566207 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.570605 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.570584 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 21 16:35:03.593782 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.593751 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8"] Jul 21 16:35:03.700622 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.700585 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.700789 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.700631 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4eac61d-9591-40a6-beb9-8480928d4c37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.700789 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.700692 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnmt6\" (UniqueName: \"kubernetes.io/projected/d4eac61d-9591-40a6-beb9-8480928d4c37-kube-api-access-fnmt6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.700789 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.700742 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.700789 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.700772 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.700974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.700845 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801350 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801322 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801524 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801361 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801524 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801396 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801524 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801458 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801524 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801502 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4eac61d-9591-40a6-beb9-8480928d4c37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801713 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801538 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnmt6\" (UniqueName: \"kubernetes.io/projected/d4eac61d-9591-40a6-beb9-8480928d4c37-kube-api-access-fnmt6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801713 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801906 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801882 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.801906 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.801902 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.803582 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.803561 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d4eac61d-9591-40a6-beb9-8480928d4c37-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.803920 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.803900 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4eac61d-9591-40a6-beb9-8480928d4c37-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.811710 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.811685 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnmt6\" (UniqueName: \"kubernetes.io/projected/d4eac61d-9591-40a6-beb9-8480928d4c37-kube-api-access-fnmt6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8\" (UID: \"d4eac61d-9591-40a6-beb9-8480928d4c37\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:03.877255 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:03.877181 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:04.011249 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:04.011219 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8"] Jul 21 16:35:04.013171 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:35:04.013142 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4eac61d_9591_40a6_beb9_8480928d4c37.slice/crio-7f35546379ac6619c997fac7e58e6b4b3f18c54fcec0ce21839dae92599fdc53 WatchSource:0}: Error finding container 7f35546379ac6619c997fac7e58e6b4b3f18c54fcec0ce21839dae92599fdc53: Status 404 returned error can't find the container with id 7f35546379ac6619c997fac7e58e6b4b3f18c54fcec0ce21839dae92599fdc53 Jul 21 16:35:04.014505 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:04.014486 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 16:35:04.543511 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:04.543479 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" event={"ID":"d4eac61d-9591-40a6-beb9-8480928d4c37","Type":"ContainerStarted","Data":"7f35546379ac6619c997fac7e58e6b4b3f18c54fcec0ce21839dae92599fdc53"} Jul 21 16:35:05.550329 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:05.550291 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" event={"ID":"d4eac61d-9591-40a6-beb9-8480928d4c37","Type":"ContainerStarted","Data":"72768bfe7de075d6566a950fcdc92dee4ec2a0345cbdceb9711c10358c081d61"} Jul 21 16:35:05.570214 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:05.570167 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" podStartSLOduration=1.408207819 podStartE2EDuration="2.570153347s" totalImagesPullingTime="1.161945528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:35:05.569153768 +0000 UTC m=+618.607989430" watchObservedRunningTime="2026-07-21 16:35:05.570153347 +0000 UTC m=+618.608989009" Jul 21 16:35:13.877668 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:13.877577 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:13.877668 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:13.877650 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:13.890049 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:13.890028 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:14.606709 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:14.606683 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8" Jul 21 16:35:17.584331 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.584295 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n"] Jul 21 16:35:17.591270 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.591243 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.594588 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.594565 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 21 16:35:17.599294 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.599273 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n"] Jul 21 16:35:17.622974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.622946 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.623088 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.623031 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.623088 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.623068 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.623162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.623096 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.623196 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.623154 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb10eec-dea6-4761-91e9-a2960b023096-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.623196 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.623185 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjhz8\" (UniqueName: \"kubernetes.io/projected/9bb10eec-dea6-4761-91e9-a2960b023096-kube-api-access-cjhz8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724002 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.723965 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724002 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724205 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724025 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb10eec-dea6-4761-91e9-a2960b023096-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724205 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjhz8\" (UniqueName: \"kubernetes.io/projected/9bb10eec-dea6-4761-91e9-a2960b023096-kube-api-access-cjhz8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724205 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724205 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724158 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724449 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724424 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724449 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724439 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.724626 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.724605 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.726321 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.726300 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9bb10eec-dea6-4761-91e9-a2960b023096-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.726461 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.726443 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9bb10eec-dea6-4761-91e9-a2960b023096-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.732580 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.732557 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjhz8\" (UniqueName: \"kubernetes.io/projected/9bb10eec-dea6-4761-91e9-a2960b023096-kube-api-access-cjhz8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ncd7n\" (UID: \"9bb10eec-dea6-4761-91e9-a2960b023096\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:17.905563 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:17.905485 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:18.047190 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:18.047156 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n"] Jul 21 16:35:18.616537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:18.616444 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" event={"ID":"9bb10eec-dea6-4761-91e9-a2960b023096","Type":"ContainerStarted","Data":"232986913e574b4ce7ca50f5424dba85180994022e18bf65a83ab50c852a49a7"} Jul 21 16:35:18.616537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:18.616495 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" event={"ID":"9bb10eec-dea6-4761-91e9-a2960b023096","Type":"ContainerStarted","Data":"d2e0d2a7e1feeb70dd0be4d22fbfd4bdc11581f60b9ea2ee20e81ac8bee1dbf8"} Jul 21 16:35:18.636973 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:18.636921 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" podStartSLOduration=1.367069116 podStartE2EDuration="1.636904278s" totalImagesPullingTime="269.835162ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:35:18.635123301 +0000 UTC m=+631.673958991" watchObservedRunningTime="2026-07-21 16:35:18.636904278 +0000 UTC m=+631.675739963" Jul 21 16:35:27.905941 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:27.905902 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:27.906373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:27.905974 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:27.919050 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:27.919024 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:35:28.681917 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:35:28.681888 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ncd7n" Jul 21 16:39:47.418732 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:39:47.418650 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:39:47.418732 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:39:47.418718 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:43:14.517872 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:14.517768 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m"] Jul 21 16:43:14.520387 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:14.518056 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" podUID="e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" containerName="manager" containerID="cri-o://276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1" gracePeriod=10 Jul 21 16:43:18.069234 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.069209 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:43:18.128879 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.128842 2557 generic.go:356] "Generic (PLEG): container finished" podID="e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" containerID="276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1" exitCode=0 Jul 21 16:43:18.128879 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.128878 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" Jul 21 16:43:18.129123 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.128918 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" event={"ID":"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c","Type":"ContainerDied","Data":"276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1"} Jul 21 16:43:18.129123 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.128973 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m" event={"ID":"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c","Type":"ContainerDied","Data":"d74cde9efd1b792d83ce7142e1a8337f1813c2046f48ed4bd4c8ab01be03c41f"} Jul 21 16:43:18.129123 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.128996 2557 scope.go:117] "RemoveContainer" containerID="276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1" Jul 21 16:43:18.138338 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.138319 2557 scope.go:117] "RemoveContainer" containerID="276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1" Jul 21 16:43:18.138589 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:43:18.138571 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1\": container with ID starting with 276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1 not found: ID does not exist" containerID="276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1" Jul 21 16:43:18.138639 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.138599 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1"} err="failed to get container status \"276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1\": rpc error: code = NotFound desc = could not find container \"276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1\": container with ID starting with 276869fe98dffb832f47468bb8c1a094b12b8470b001024c4549067cdceee2c1 not found: ID does not exist" Jul 21 16:43:18.188075 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.188018 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97j5z\" (UniqueName: \"kubernetes.io/projected/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-kube-api-access-97j5z\") pod \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " Jul 21 16:43:18.188177 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.188133 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-extensions-socket-volume\") pod \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\" (UID: \"e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c\") " Jul 21 16:43:18.188477 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.188448 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" (UID: "e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:43:18.189929 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.189904 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-kube-api-access-97j5z" (OuterVolumeSpecName: "kube-api-access-97j5z") pod "e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" (UID: "e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c"). InnerVolumeSpecName "kube-api-access-97j5z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:43:18.289672 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.289645 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-extensions-socket-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:43:18.289672 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.289673 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97j5z\" (UniqueName: \"kubernetes.io/projected/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c-kube-api-access-97j5z\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:43:18.451305 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.451246 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m"] Jul 21 16:43:18.456607 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:18.456585 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xks6m"] Jul 21 16:43:19.489348 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:43:19.489315 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" path="/var/lib/kubelet/pods/e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c/volumes" Jul 21 16:44:47.457289 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:44:47.457255 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:44:47.461119 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:44:47.461094 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:45:00.150166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.150133 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9"] Jul 21 16:45:00.150558 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.150545 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" containerName="manager" Jul 21 16:45:00.150601 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.150559 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" containerName="manager" Jul 21 16:45:00.150638 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.150633 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e867ca42-c496-4dfe-8ab4-9f0bdf9cfb0c" containerName="manager" Jul 21 16:45:00.153716 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.153698 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9" Jul 21 16:45:00.156628 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.156605 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zwrvn\"" Jul 21 16:45:00.162163 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.162135 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9"] Jul 21 16:45:00.164266 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.164244 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9" Jul 21 16:45:00.504973 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.504944 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9"] Jul 21 16:45:00.506493 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:45:00.506468 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf498052_6027_467b_a649_d77cece3769e.slice/crio-64b7b40467673363e12790b653c546500c70d55a8a341751b13985894ab1755b WatchSource:0}: Error finding container 64b7b40467673363e12790b653c546500c70d55a8a341751b13985894ab1755b: Status 404 returned error can't find the container with id 64b7b40467673363e12790b653c546500c70d55a8a341751b13985894ab1755b Jul 21 16:45:00.508295 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.508278 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 16:45:00.643419 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:00.643386 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9" event={"ID":"df498052-6027-467b-a649-d77cece3769e","Type":"ContainerStarted","Data":"64b7b40467673363e12790b653c546500c70d55a8a341751b13985894ab1755b"} Jul 21 16:45:03.659900 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:03.659866 2557 generic.go:356] "Generic (PLEG): container finished" podID="df498052-6027-467b-a649-d77cece3769e" containerID="8fae902c0a8ad66bbf4cbc5909e2a4b090e42179988fc4005b46e75f5738030b" exitCode=0 Jul 21 16:45:03.660225 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:03.659956 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9" event={"ID":"df498052-6027-467b-a649-d77cece3769e","Type":"ContainerDied","Data":"8fae902c0a8ad66bbf4cbc5909e2a4b090e42179988fc4005b46e75f5738030b"} Jul 21 16:45:04.797601 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:04.797572 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9" Jul 21 16:45:05.671637 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:05.671606 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9" Jul 21 16:45:05.671806 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:05.671647 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744205-6gcw9" event={"ID":"df498052-6027-467b-a649-d77cece3769e","Type":"ContainerDied","Data":"64b7b40467673363e12790b653c546500c70d55a8a341751b13985894ab1755b"} Jul 21 16:45:05.671806 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:45:05.671694 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="64b7b40467673363e12790b653c546500c70d55a8a341751b13985894ab1755b" Jul 21 16:49:47.496213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:49:47.496182 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:49:47.498807 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:49:47.498784 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:51:53.796794 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.796689 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt"] Jul 21 16:51:53.797290 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.797207 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df498052-6027-467b-a649-d77cece3769e" containerName="cleanup" Jul 21 16:51:53.797290 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.797221 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="df498052-6027-467b-a649-d77cece3769e" containerName="cleanup" Jul 21 16:51:53.797290 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.797292 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="df498052-6027-467b-a649-d77cece3769e" containerName="cleanup" Jul 21 16:51:53.800186 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.800170 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" Jul 21 16:51:53.804082 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.804060 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 16:51:53.804205 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.804098 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zwrvn\"" Jul 21 16:51:53.810427 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.810403 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt"] Jul 21 16:51:53.856797 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.856743 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1264786-9236-4d05-beb7-a44576b5aa12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-989b148b-4xbnt\" (UID: \"a1264786-9236-4d05-beb7-a44576b5aa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" Jul 21 16:51:53.958079 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.958046 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1264786-9236-4d05-beb7-a44576b5aa12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-989b148b-4xbnt\" (UID: \"a1264786-9236-4d05-beb7-a44576b5aa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" Jul 21 16:51:53.958716 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:53.958692 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1264786-9236-4d05-beb7-a44576b5aa12-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-989b148b-4xbnt\" (UID: \"a1264786-9236-4d05-beb7-a44576b5aa12\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" Jul 21 16:51:54.111418 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:54.111332 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" Jul 21 16:51:54.250148 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:54.250117 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt"] Jul 21 16:51:54.253799 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:51:54.253775 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1264786_9236_4d05_beb7_a44576b5aa12.slice/crio-23cb4869310fe2c6ef98aad50da12aca85b393696db183090f4cd368ef90345b WatchSource:0}: Error finding container 23cb4869310fe2c6ef98aad50da12aca85b393696db183090f4cd368ef90345b: Status 404 returned error can't find the container with id 23cb4869310fe2c6ef98aad50da12aca85b393696db183090f4cd368ef90345b Jul 21 16:51:54.760269 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:54.760227 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" event={"ID":"a1264786-9236-4d05-beb7-a44576b5aa12","Type":"ContainerStarted","Data":"3aab6829463c1da2ecd9a07c9d9a0a6e6bf445abfc36aec16616245c99fdf76e"} Jul 21 16:51:54.760454 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:54.760297 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" event={"ID":"a1264786-9236-4d05-beb7-a44576b5aa12","Type":"ContainerStarted","Data":"23cb4869310fe2c6ef98aad50da12aca85b393696db183090f4cd368ef90345b"} Jul 21 16:51:54.777398 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:54.777349 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" podStartSLOduration=1.7773322999999999 podStartE2EDuration="1.7773323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:51:54.775611008 +0000 UTC m=+1627.814446670" watchObservedRunningTime="2026-07-21 16:51:54.7773323 +0000 UTC m=+1627.816168007" Jul 21 16:51:55.766498 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:55.766463 2557 generic.go:356] "Generic (PLEG): container finished" podID="a1264786-9236-4d05-beb7-a44576b5aa12" containerID="3aab6829463c1da2ecd9a07c9d9a0a6e6bf445abfc36aec16616245c99fdf76e" exitCode=0 Jul 21 16:51:55.766954 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:55.766508 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" event={"ID":"a1264786-9236-4d05-beb7-a44576b5aa12","Type":"ContainerDied","Data":"3aab6829463c1da2ecd9a07c9d9a0a6e6bf445abfc36aec16616245c99fdf76e"} Jul 21 16:51:56.912207 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:56.912175 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" Jul 21 16:51:56.987174 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:56.987138 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1264786-9236-4d05-beb7-a44576b5aa12-maas-api-service-ca\") pod \"a1264786-9236-4d05-beb7-a44576b5aa12\" (UID: \"a1264786-9236-4d05-beb7-a44576b5aa12\") " Jul 21 16:51:56.987506 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:56.987480 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1264786-9236-4d05-beb7-a44576b5aa12-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a1264786-9236-4d05-beb7-a44576b5aa12" (UID: "a1264786-9236-4d05-beb7-a44576b5aa12"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:51:57.087870 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:57.087778 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1264786-9236-4d05-beb7-a44576b5aa12-maas-api-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:51:57.777974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:57.777881 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" Jul 21 16:51:57.777974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:57.777918 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt" event={"ID":"a1264786-9236-4d05-beb7-a44576b5aa12","Type":"ContainerDied","Data":"23cb4869310fe2c6ef98aad50da12aca85b393696db183090f4cd368ef90345b"} Jul 21 16:51:57.777974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:51:57.777968 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23cb4869310fe2c6ef98aad50da12aca85b393696db183090f4cd368ef90345b" Jul 21 16:52:09.014167 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:09.014128 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt"] Jul 21 16:52:09.023406 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:09.023372 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-989b148b-4xbnt"] Jul 21 16:52:09.489087 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:09.489055 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1264786-9236-4d05-beb7-a44576b5aa12" path="/var/lib/kubelet/pods/a1264786-9236-4d05-beb7-a44576b5aa12/volumes" Jul 21 16:52:12.777314 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.777279 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b"] Jul 21 16:52:12.777750 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.777735 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1264786-9236-4d05-beb7-a44576b5aa12" containerName="revoke-keys" Jul 21 16:52:12.777793 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.777753 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1264786-9236-4d05-beb7-a44576b5aa12" containerName="revoke-keys" Jul 21 16:52:12.777849 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.777836 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1264786-9236-4d05-beb7-a44576b5aa12" containerName="revoke-keys" Jul 21 16:52:12.781268 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.781251 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.785745 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.785721 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-512e02f1-serving-cert\"" Jul 21 16:52:12.785894 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.785740 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 16:52:12.785894 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.785776 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 16:52:12.785894 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.785733 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9vn8\"" Jul 21 16:52:12.791273 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.791251 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b"] Jul 21 16:52:12.835649 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.835612 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/996a2315-4ce2-40c5-9643-84a31bf4e605-openshift-service-ca\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.835869 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.835682 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5xlk\" (UniqueName: \"kubernetes.io/projected/996a2315-4ce2-40c5-9643-84a31bf4e605-kube-api-access-b5xlk\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.835869 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.835778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.936399 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.936364 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5xlk\" (UniqueName: \"kubernetes.io/projected/996a2315-4ce2-40c5-9643-84a31bf4e605-kube-api-access-b5xlk\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.936572 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.936431 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.936572 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.936467 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/996a2315-4ce2-40c5-9643-84a31bf4e605-openshift-service-ca\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.936664 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:52:12.936568 2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-serving-cert: secret "maas-api-e2e-migrate-512e02f1-serving-cert" not found Jul 21 16:52:12.936664 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:52:12.936625 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls podName:996a2315-4ce2-40c5-9643-84a31bf4e605 nodeName:}" failed. No retries permitted until 2026-07-21 16:52:13.436609213 +0000 UTC m=+1646.475444858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls") pod "maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" (UID: "996a2315-4ce2-40c5-9643-84a31bf4e605") : secret "maas-api-e2e-migrate-512e02f1-serving-cert" not found Jul 21 16:52:12.937084 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.937068 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/996a2315-4ce2-40c5-9643-84a31bf4e605-openshift-service-ca\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:12.950612 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:12.950581 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5xlk\" (UniqueName: \"kubernetes.io/projected/996a2315-4ce2-40c5-9643-84a31bf4e605-kube-api-access-b5xlk\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:13.442260 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:13.442227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:13.444691 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:13.444671 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls\") pod \"maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:13.692843 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:13.692735 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:13.841621 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:13.841597 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b"] Jul 21 16:52:13.844091 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:52:13.844051 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996a2315_4ce2_40c5_9643_84a31bf4e605.slice/crio-a354037717435eadc162dc5761bfbc27db9234e835e981158e82f8a9ace4508b WatchSource:0}: Error finding container a354037717435eadc162dc5761bfbc27db9234e835e981158e82f8a9ace4508b: Status 404 returned error can't find the container with id a354037717435eadc162dc5761bfbc27db9234e835e981158e82f8a9ace4508b Jul 21 16:52:13.845635 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:13.845615 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 16:52:13.859535 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:13.859507 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" event={"ID":"996a2315-4ce2-40c5-9643-84a31bf4e605","Type":"ContainerStarted","Data":"a354037717435eadc162dc5761bfbc27db9234e835e981158e82f8a9ace4508b"} Jul 21 16:52:16.878249 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:16.878205 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" event={"ID":"996a2315-4ce2-40c5-9643-84a31bf4e605","Type":"ContainerStarted","Data":"aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16"} Jul 21 16:52:16.878698 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:16.878275 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:16.897373 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:16.897324 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" podStartSLOduration=2.494149061 podStartE2EDuration="4.897307392s" totalImagesPullingTime="2.403158331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:52:16.895921645 +0000 UTC m=+1649.934757307" watchObservedRunningTime="2026-07-21 16:52:16.897307392 +0000 UTC m=+1649.936143081" Jul 21 16:52:22.889700 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:22.889672 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:37.735130 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:37.735096 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk"] Jul 21 16:52:37.741600 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:37.741577 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" Jul 21 16:52:37.744614 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:37.744597 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zwrvn\"" Jul 21 16:52:37.751539 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:37.751514 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk"] Jul 21 16:52:37.874492 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:37.874460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5adadd-4113-4706-81c6-4db841dceab2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk\" (UID: \"ba5adadd-4113-4706-81c6-4db841dceab2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" Jul 21 16:52:37.975227 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:37.975184 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5adadd-4113-4706-81c6-4db841dceab2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk\" (UID: \"ba5adadd-4113-4706-81c6-4db841dceab2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" Jul 21 16:52:37.975835 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:37.975802 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5adadd-4113-4706-81c6-4db841dceab2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk\" (UID: \"ba5adadd-4113-4706-81c6-4db841dceab2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" Jul 21 16:52:38.052513 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:38.052422 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" Jul 21 16:52:38.199723 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:38.199693 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk"] Jul 21 16:52:38.200381 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:52:38.200357 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba5adadd_4113_4706_81c6_4db841dceab2.slice/crio-208d3a3a96c8846b7056890939e2afbe7d821b279694c1f514d61ef01fb692d9 WatchSource:0}: Error finding container 208d3a3a96c8846b7056890939e2afbe7d821b279694c1f514d61ef01fb692d9: Status 404 returned error can't find the container with id 208d3a3a96c8846b7056890939e2afbe7d821b279694c1f514d61ef01fb692d9 Jul 21 16:52:38.990748 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:38.990713 2557 generic.go:356] "Generic (PLEG): container finished" podID="ba5adadd-4113-4706-81c6-4db841dceab2" containerID="2c6de9e3d5d5210c24842508d22b275cb610921694d20276f41fe7cf5b21e753" exitCode=0 Jul 21 16:52:38.991177 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:38.990787 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" event={"ID":"ba5adadd-4113-4706-81c6-4db841dceab2","Type":"ContainerDied","Data":"2c6de9e3d5d5210c24842508d22b275cb610921694d20276f41fe7cf5b21e753"} Jul 21 16:52:38.991177 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:38.990848 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" event={"ID":"ba5adadd-4113-4706-81c6-4db841dceab2","Type":"ContainerStarted","Data":"208d3a3a96c8846b7056890939e2afbe7d821b279694c1f514d61ef01fb692d9"} Jul 21 16:52:40.127765 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:40.127742 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" Jul 21 16:52:40.294204 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:40.294116 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5adadd-4113-4706-81c6-4db841dceab2-maas-api-service-ca\") pod \"ba5adadd-4113-4706-81c6-4db841dceab2\" (UID: \"ba5adadd-4113-4706-81c6-4db841dceab2\") " Jul 21 16:52:40.294529 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:40.294510 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba5adadd-4113-4706-81c6-4db841dceab2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ba5adadd-4113-4706-81c6-4db841dceab2" (UID: "ba5adadd-4113-4706-81c6-4db841dceab2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:52:40.395657 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:40.395617 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ba5adadd-4113-4706-81c6-4db841dceab2-maas-api-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:52:41.003078 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:41.003043 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" event={"ID":"ba5adadd-4113-4706-81c6-4db841dceab2","Type":"ContainerDied","Data":"208d3a3a96c8846b7056890939e2afbe7d821b279694c1f514d61ef01fb692d9"} Jul 21 16:52:41.003257 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:41.003119 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="208d3a3a96c8846b7056890939e2afbe7d821b279694c1f514d61ef01fb692d9" Jul 21 16:52:41.003257 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:41.003054 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk" Jul 21 16:52:48.524698 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.524664 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b"] Jul 21 16:52:48.525178 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.524883 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" podUID="996a2315-4ce2-40c5-9643-84a31bf4e605" containerName="maas-api" containerID="cri-o://aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16" gracePeriod=30 Jul 21 16:52:48.785087 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.785032 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:48.973014 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.972971 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5xlk\" (UniqueName: \"kubernetes.io/projected/996a2315-4ce2-40c5-9643-84a31bf4e605-kube-api-access-b5xlk\") pod \"996a2315-4ce2-40c5-9643-84a31bf4e605\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " Jul 21 16:52:48.973174 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.973112 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/996a2315-4ce2-40c5-9643-84a31bf4e605-openshift-service-ca\") pod \"996a2315-4ce2-40c5-9643-84a31bf4e605\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " Jul 21 16:52:48.973174 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.973146 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls\") pod \"996a2315-4ce2-40c5-9643-84a31bf4e605\" (UID: \"996a2315-4ce2-40c5-9643-84a31bf4e605\") " Jul 21 16:52:48.973503 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.973468 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/996a2315-4ce2-40c5-9643-84a31bf4e605-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "996a2315-4ce2-40c5-9643-84a31bf4e605" (UID: "996a2315-4ce2-40c5-9643-84a31bf4e605"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:52:48.975260 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.975228 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "996a2315-4ce2-40c5-9643-84a31bf4e605" (UID: "996a2315-4ce2-40c5-9643-84a31bf4e605"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 16:52:48.975358 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:48.975285 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/996a2315-4ce2-40c5-9643-84a31bf4e605-kube-api-access-b5xlk" (OuterVolumeSpecName: "kube-api-access-b5xlk") pod "996a2315-4ce2-40c5-9643-84a31bf4e605" (UID: "996a2315-4ce2-40c5-9643-84a31bf4e605"). InnerVolumeSpecName "kube-api-access-b5xlk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:52:49.045018 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.044929 2557 generic.go:356] "Generic (PLEG): container finished" podID="996a2315-4ce2-40c5-9643-84a31bf4e605" containerID="aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16" exitCode=0 Jul 21 16:52:49.045018 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.044981 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" Jul 21 16:52:49.045239 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.045016 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" event={"ID":"996a2315-4ce2-40c5-9643-84a31bf4e605","Type":"ContainerDied","Data":"aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16"} Jul 21 16:52:49.045239 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.045068 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b" event={"ID":"996a2315-4ce2-40c5-9643-84a31bf4e605","Type":"ContainerDied","Data":"a354037717435eadc162dc5761bfbc27db9234e835e981158e82f8a9ace4508b"} Jul 21 16:52:49.045239 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.045090 2557 scope.go:117] "RemoveContainer" containerID="aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16" Jul 21 16:52:49.055528 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.055512 2557 scope.go:117] "RemoveContainer" containerID="aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16" Jul 21 16:52:49.055793 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:52:49.055771 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16\": container with ID starting with aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16 not found: ID does not exist" containerID="aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16" Jul 21 16:52:49.055879 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.055803 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16"} err="failed to get container status \"aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16\": rpc error: code = NotFound desc = could not find container \"aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16\": container with ID starting with aeeeeee73c2623277759d35dc2a89d844ab415337d134cfb60efc27791412a16 not found: ID does not exist" Jul 21 16:52:49.074568 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.074546 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/996a2315-4ce2-40c5-9643-84a31bf4e605-openshift-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:52:49.074568 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.074568 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/996a2315-4ce2-40c5-9643-84a31bf4e605-maas-api-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:52:49.074699 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.074578 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5xlk\" (UniqueName: \"kubernetes.io/projected/996a2315-4ce2-40c5-9643-84a31bf4e605-kube-api-access-b5xlk\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:52:49.083452 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.083431 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b"] Jul 21 16:52:49.088250 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.088229 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-512e02f1-5d44b4bf9-5zb8b"] Jul 21 16:52:49.490424 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:49.490387 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="996a2315-4ce2-40c5-9643-84a31bf4e605" path="/var/lib/kubelet/pods/996a2315-4ce2-40c5-9643-84a31bf4e605/volumes" Jul 21 16:52:53.043979 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:53.043939 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk"] Jul 21 16:52:53.053882 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:53.053847 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-512e02f1-gz9mk"] Jul 21 16:52:53.488986 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:53.488948 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba5adadd-4113-4706-81c6-4db841dceab2" path="/var/lib/kubelet/pods/ba5adadd-4113-4706-81c6-4db841dceab2/volumes" Jul 21 16:52:54.227487 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.227453 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n"] Jul 21 16:52:54.228076 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.228057 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="996a2315-4ce2-40c5-9643-84a31bf4e605" containerName="maas-api" Jul 21 16:52:54.228160 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.228079 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="996a2315-4ce2-40c5-9643-84a31bf4e605" containerName="maas-api" Jul 21 16:52:54.228160 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.228096 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba5adadd-4113-4706-81c6-4db841dceab2" containerName="revoke-keys" Jul 21 16:52:54.228160 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.228106 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba5adadd-4113-4706-81c6-4db841dceab2" containerName="revoke-keys" Jul 21 16:52:54.228301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.228202 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba5adadd-4113-4706-81c6-4db841dceab2" containerName="revoke-keys" Jul 21 16:52:54.228301 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.228221 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="996a2315-4ce2-40c5-9643-84a31bf4e605" containerName="maas-api" Jul 21 16:52:54.232810 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.232787 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.236526 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.236503 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6ccfd18f-openshift-default-dockercfg-v8bss\"" Jul 21 16:52:54.253249 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.253225 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n"] Jul 21 16:52:54.319087 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319046 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-certs\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319265 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319097 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-credential-socket\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319265 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-socket\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319265 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319162 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-envoy\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319265 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319226 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-podinfo\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319265 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-token\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319453 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319320 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c564845-0c4d-49a3-b957-3b344a9b955e-istiod-ca-cert\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319453 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319355 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65244\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-kube-api-access-65244\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.319453 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.319392 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-data\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.419922 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.419879 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c564845-0c4d-49a3-b957-3b344a9b955e-istiod-ca-cert\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.419922 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.419921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65244\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-kube-api-access-65244\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.419973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-data\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-certs\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-credential-socket\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420125 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-socket\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-envoy\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420412 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420182 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-podinfo\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420412 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420217 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-token\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420535 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420419 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-data\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420662 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420602 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-credential-socket\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420662 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420633 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-certs\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420747 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420698 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c564845-0c4d-49a3-b957-3b344a9b955e-istiod-ca-cert\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.420747 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.420707 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-socket\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.422593 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.422568 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-envoy\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.422864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.422842 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-podinfo\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.431174 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.431150 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-token\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.433713 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.433690 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65244\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-kube-api-access-65244\") pod \"e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.543902 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.543810 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:54.704933 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:52:54.704901 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c564845_0c4d_49a3_b957_3b344a9b955e.slice/crio-900dd9df1d8c430cf8870861451423940646533c08b0aece90b015eebe18205e WatchSource:0}: Error finding container 900dd9df1d8c430cf8870861451423940646533c08b0aece90b015eebe18205e: Status 404 returned error can't find the container with id 900dd9df1d8c430cf8870861451423940646533c08b0aece90b015eebe18205e Jul 21 16:52:54.706003 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:54.705979 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n"] Jul 21 16:52:55.078463 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:55.078422 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" event={"ID":"2c564845-0c4d-49a3-b957-3b344a9b955e","Type":"ContainerStarted","Data":"900dd9df1d8c430cf8870861451423940646533c08b0aece90b015eebe18205e"} Jul 21 16:52:57.570994 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:57.570957 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:52:57.571305 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:57.571030 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:52:57.571305 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:57.571054 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:52:58.098929 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:58.098889 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" event={"ID":"2c564845-0c4d-49a3-b957-3b344a9b955e","Type":"ContainerStarted","Data":"95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433"} Jul 21 16:52:58.125855 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:58.125787 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" podStartSLOduration=1.262136353 podStartE2EDuration="4.125771372s" totalImagesPullingTime="2.863635019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:52:58.12406153 +0000 UTC m=+1691.162897196" watchObservedRunningTime="2026-07-21 16:52:58.125771372 +0000 UTC m=+1691.164607033" Jul 21 16:52:58.543996 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:58.543958 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:58.548898 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:58.548873 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:59.103885 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.103854 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:59.104769 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.104750 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:52:59.783088 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.783053 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l"] Jul 21 16:52:59.786924 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.786908 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.794245 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.794205 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-6ccfd18f-serving-cert\"" Jul 21 16:52:59.794371 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.794352 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9vn8\"" Jul 21 16:52:59.795408 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.795392 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 16:52:59.795489 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.795396 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 16:52:59.815389 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.815359 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l"] Jul 21 16:52:59.865412 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.865376 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-openshift-service-ca\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.865574 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.865442 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.865574 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.865462 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwszr\" (UniqueName: \"kubernetes.io/projected/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-kube-api-access-dwszr\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.966283 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.966245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-openshift-service-ca\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.966434 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.966314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.966434 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.966342 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwszr\" (UniqueName: \"kubernetes.io/projected/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-kube-api-access-dwszr\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.966558 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:52:59.966458 2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-serving-cert: secret "maas-api-e2e-ait-6ccfd18f-serving-cert" not found Jul 21 16:52:59.966558 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:52:59.966542 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls podName:8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59 nodeName:}" failed. No retries permitted until 2026-07-21 16:53:00.466519121 +0000 UTC m=+1693.505354761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls") pod "maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" (UID: "8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59") : secret "maas-api-e2e-ait-6ccfd18f-serving-cert" not found Jul 21 16:52:59.967034 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.967017 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-openshift-service-ca\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:52:59.982070 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:52:59.982049 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwszr\" (UniqueName: \"kubernetes.io/projected/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-kube-api-access-dwszr\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:00.471359 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:00.471324 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:00.473804 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:00.473779 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls\") pod \"maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:00.697957 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:00.697922 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:00.843564 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:00.843534 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l"] Jul 21 16:53:00.845005 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:53:00.844968 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b802c8a_c6bf_4aca_b0c6_b7ed19b68d59.slice/crio-766c65b27b46a335cb091baa1b5327190c33aa63ff956c47cb7aa1ac0b10e83c WatchSource:0}: Error finding container 766c65b27b46a335cb091baa1b5327190c33aa63ff956c47cb7aa1ac0b10e83c: Status 404 returned error can't find the container with id 766c65b27b46a335cb091baa1b5327190c33aa63ff956c47cb7aa1ac0b10e83c Jul 21 16:53:01.115518 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:01.115429 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" event={"ID":"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59","Type":"ContainerStarted","Data":"766c65b27b46a335cb091baa1b5327190c33aa63ff956c47cb7aa1ac0b10e83c"} Jul 21 16:53:02.123127 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:02.123078 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" event={"ID":"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59","Type":"ContainerStarted","Data":"b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d"} Jul 21 16:53:02.123537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:02.123279 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:02.149444 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:02.149385 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" podStartSLOduration=2.699516841 podStartE2EDuration="3.14936994s" totalImagesPullingTime="449.853099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:53:02.147198237 +0000 UTC m=+1695.186033898" watchObservedRunningTime="2026-07-21 16:53:02.14936994 +0000 UTC m=+1695.188205601" Jul 21 16:53:08.144166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:08.144130 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:12.183633 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:12.183533 2557 generic.go:356] "Generic (PLEG): container finished" podID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerID="95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433" exitCode=0 Jul 21 16:53:12.183633 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:12.183609 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" event={"ID":"2c564845-0c4d-49a3-b957-3b344a9b955e","Type":"ContainerDied","Data":"95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433"} Jul 21 16:53:12.184083 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:12.184016 2557 scope.go:117] "RemoveContainer" containerID="95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433" Jul 21 16:53:12.185947 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:12.185918 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:12.186035 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:12.185984 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:12.186087 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:12.186034 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:13.191195 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:13.191160 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" event={"ID":"2c564845-0c4d-49a3-b957-3b344a9b955e","Type":"ContainerStarted","Data":"90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10"} Jul 21 16:53:13.547272 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:13.547228 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:53:13.548485 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:13.548457 2557 patch_prober.go:28] interesting pod/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 16:53:13.548622 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:13.548557 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 16:53:14.104722 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:14.104688 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:53:14.545066 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:14.545027 2557 patch_prober.go:28] interesting pod/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 16:53:14.545569 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:14.545098 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 16:53:15.544839 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:15.544782 2557 patch_prober.go:28] interesting pod/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 16:53:15.545049 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:15.544871 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 16:53:16.544964 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:16.544922 2557 patch_prober.go:28] interesting pod/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 21 16:53:16.545478 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:16.544995 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 21 16:53:17.548020 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:17.547989 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:53:17.548864 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:17.548845 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:53:25.309737 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.309701 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq"] Jul 21 16:53:25.317874 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.317855 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" Jul 21 16:53:25.322833 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.322800 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zwrvn\"" Jul 21 16:53:25.328227 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.328203 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq"] Jul 21 16:53:25.391759 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.391725 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b91f0066-fe35-4fa0-8ba0-41ff73e96f42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq\" (UID: \"b91f0066-fe35-4fa0-8ba0-41ff73e96f42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" Jul 21 16:53:25.492885 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.492852 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b91f0066-fe35-4fa0-8ba0-41ff73e96f42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq\" (UID: \"b91f0066-fe35-4fa0-8ba0-41ff73e96f42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" Jul 21 16:53:25.493448 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.493420 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b91f0066-fe35-4fa0-8ba0-41ff73e96f42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq\" (UID: \"b91f0066-fe35-4fa0-8ba0-41ff73e96f42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" Jul 21 16:53:25.630531 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.630444 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" Jul 21 16:53:25.775810 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:53:25.775776 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb91f0066_fe35_4fa0_8ba0_41ff73e96f42.slice/crio-9afedc188bf105271467ea5b749e4ef2b8486fd1e32edba3cfc0464da4bdb44e WatchSource:0}: Error finding container 9afedc188bf105271467ea5b749e4ef2b8486fd1e32edba3cfc0464da4bdb44e: Status 404 returned error can't find the container with id 9afedc188bf105271467ea5b749e4ef2b8486fd1e32edba3cfc0464da4bdb44e Jul 21 16:53:25.775944 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:25.775887 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq"] Jul 21 16:53:26.264419 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:26.264385 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" event={"ID":"b91f0066-fe35-4fa0-8ba0-41ff73e96f42","Type":"ContainerStarted","Data":"fdb49be2b5083a150a3e773fd5d1dfb899d5ad5bf422977e29cc50fbc4211d37"} Jul 21 16:53:26.264590 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:26.264435 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" event={"ID":"b91f0066-fe35-4fa0-8ba0-41ff73e96f42","Type":"ContainerStarted","Data":"9afedc188bf105271467ea5b749e4ef2b8486fd1e32edba3cfc0464da4bdb44e"} Jul 21 16:53:26.287070 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:26.287025 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" podStartSLOduration=1.287005702 podStartE2EDuration="1.287005702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:53:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:53:26.285321051 +0000 UTC m=+1719.324156718" watchObservedRunningTime="2026-07-21 16:53:26.287005702 +0000 UTC m=+1719.325841407" Jul 21 16:53:27.272692 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:27.272658 2557 generic.go:356] "Generic (PLEG): container finished" podID="b91f0066-fe35-4fa0-8ba0-41ff73e96f42" containerID="fdb49be2b5083a150a3e773fd5d1dfb899d5ad5bf422977e29cc50fbc4211d37" exitCode=0 Jul 21 16:53:27.273095 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:27.272735 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" event={"ID":"b91f0066-fe35-4fa0-8ba0-41ff73e96f42","Type":"ContainerDied","Data":"fdb49be2b5083a150a3e773fd5d1dfb899d5ad5bf422977e29cc50fbc4211d37"} Jul 21 16:53:28.415968 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:28.415944 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" Jul 21 16:53:28.521802 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:28.521774 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b91f0066-fe35-4fa0-8ba0-41ff73e96f42-maas-api-service-ca\") pod \"b91f0066-fe35-4fa0-8ba0-41ff73e96f42\" (UID: \"b91f0066-fe35-4fa0-8ba0-41ff73e96f42\") " Jul 21 16:53:28.522109 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:28.522089 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b91f0066-fe35-4fa0-8ba0-41ff73e96f42-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b91f0066-fe35-4fa0-8ba0-41ff73e96f42" (UID: "b91f0066-fe35-4fa0-8ba0-41ff73e96f42"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:53:28.623513 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:28.623431 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b91f0066-fe35-4fa0-8ba0-41ff73e96f42-maas-api-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:29.284636 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:29.284602 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" event={"ID":"b91f0066-fe35-4fa0-8ba0-41ff73e96f42","Type":"ContainerDied","Data":"9afedc188bf105271467ea5b749e4ef2b8486fd1e32edba3cfc0464da4bdb44e"} Jul 21 16:53:29.284636 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:29.284631 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq" Jul 21 16:53:29.284927 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:29.284650 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9afedc188bf105271467ea5b749e4ef2b8486fd1e32edba3cfc0464da4bdb44e" Jul 21 16:53:35.762451 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:35.762418 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l"] Jul 21 16:53:35.762870 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:35.762633 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" podUID="8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" containerName="maas-api" containerID="cri-o://b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d" gracePeriod=30 Jul 21 16:53:36.038116 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.038052 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:36.192455 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.192426 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-openshift-service-ca\") pod \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " Jul 21 16:53:36.192627 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.192514 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls\") pod \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " Jul 21 16:53:36.192627 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.192536 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwszr\" (UniqueName: \"kubernetes.io/projected/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-kube-api-access-dwszr\") pod \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\" (UID: \"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59\") " Jul 21 16:53:36.192814 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.192786 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" (UID: "8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:53:36.194474 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.194455 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" (UID: "8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 16:53:36.194685 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.194664 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-kube-api-access-dwszr" (OuterVolumeSpecName: "kube-api-access-dwszr") pod "8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" (UID: "8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59"). InnerVolumeSpecName "kube-api-access-dwszr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:53:36.293317 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.293219 2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-openshift-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:36.293317 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.293264 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-maas-api-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:36.293317 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.293278 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwszr\" (UniqueName: \"kubernetes.io/projected/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59-kube-api-access-dwszr\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:36.323133 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.323099 2557 generic.go:356] "Generic (PLEG): container finished" podID="8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" containerID="b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d" exitCode=0 Jul 21 16:53:36.323286 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.323143 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" Jul 21 16:53:36.323286 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.323191 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" event={"ID":"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59","Type":"ContainerDied","Data":"b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d"} Jul 21 16:53:36.323286 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.323249 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l" event={"ID":"8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59","Type":"ContainerDied","Data":"766c65b27b46a335cb091baa1b5327190c33aa63ff956c47cb7aa1ac0b10e83c"} Jul 21 16:53:36.323286 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.323274 2557 scope.go:117] "RemoveContainer" containerID="b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d" Jul 21 16:53:36.333426 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.333405 2557 scope.go:117] "RemoveContainer" containerID="b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d" Jul 21 16:53:36.333738 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:53:36.333711 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d\": container with ID starting with b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d not found: ID does not exist" containerID="b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d" Jul 21 16:53:36.333791 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.333751 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d"} err="failed to get container status \"b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d\": rpc error: code = NotFound desc = could not find container \"b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d\": container with ID starting with b02b526ff87a613ca795755e11c291dac4dccacd72a224d6857caaaba3f2694d not found: ID does not exist" Jul 21 16:53:36.347507 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.347483 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l"] Jul 21 16:53:36.352167 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:36.352146 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-6ccfd18f-7c6bcf8d99-t4z2l"] Jul 21 16:53:37.490905 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:37.490861 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" path="/var/lib/kubelet/pods/8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59/volumes" Jul 21 16:53:40.399388 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:40.399353 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq"] Jul 21 16:53:40.406176 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:40.406151 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-6ccfd18f-kf7tq"] Jul 21 16:53:41.489075 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:41.489043 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b91f0066-fe35-4fa0-8ba0-41ff73e96f42" path="/var/lib/kubelet/pods/b91f0066-fe35-4fa0-8ba0-41ff73e96f42/volumes" Jul 21 16:53:41.579989 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:41.579955 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n"] Jul 21 16:53:41.580350 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:41.580315 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" containerID="cri-o://90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10" gracePeriod=30 Jul 21 16:53:42.431499 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.431454 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp"] Jul 21 16:53:42.431902 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.431885 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b91f0066-fe35-4fa0-8ba0-41ff73e96f42" containerName="revoke-keys" Jul 21 16:53:42.431902 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.431903 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b91f0066-fe35-4fa0-8ba0-41ff73e96f42" containerName="revoke-keys" Jul 21 16:53:42.432031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.431924 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" containerName="maas-api" Jul 21 16:53:42.432031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.431934 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" containerName="maas-api" Jul 21 16:53:42.432031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.432009 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b91f0066-fe35-4fa0-8ba0-41ff73e96f42" containerName="revoke-keys" Jul 21 16:53:42.432031 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.432020 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b802c8a-c6bf-4aca-b0c6-b7ed19b68d59" containerName="maas-api" Jul 21 16:53:42.436706 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.436683 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.440330 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.440310 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9f938885-openshift-default-dockercfg-qjmzx\"" Jul 21 16:53:42.466022 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.465981 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp"] Jul 21 16:53:42.551685 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.551649 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-socket\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.551685 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.551688 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-certs\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.552129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.551706 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-credential-socket\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.552129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.551767 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istiod-ca-cert\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.552129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.551872 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-envoy\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.552129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.551896 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxpzf\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-kube-api-access-qxpzf\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.552129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.551975 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-token\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.552129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.552029 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-data\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.552129 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.552046 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-podinfo\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.652969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.652931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-socket\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.652969 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.652974 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-certs\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653201 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.652997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-credential-socket\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653201 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istiod-ca-cert\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653201 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653172 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-envoy\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653365 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653202 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxpzf\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-kube-api-access-qxpzf\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653365 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653281 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-token\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653365 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653348 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-data\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653363 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-socket\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653372 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-podinfo\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653407 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-certs\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653444 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-credential-socket\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653762 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653694 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-data\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.653929 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.653908 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istiod-ca-cert\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.655492 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.655474 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-envoy\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.655649 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.655631 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-podinfo\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.663704 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.663681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxpzf\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-kube-api-access-qxpzf\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.668256 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.668231 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-token\") pod \"e2e-derive-9f938885-openshift-default-5d49c648bc-svffp\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.747644 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.747613 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:42.908345 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.908311 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp"] Jul 21 16:53:42.912878 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.912841 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:42.913035 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.912914 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:42.913081 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:42.913056 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:43.367518 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:43.367483 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" event={"ID":"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c","Type":"ContainerStarted","Data":"500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419"} Jul 21 16:53:43.367685 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:43.367542 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" event={"ID":"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c","Type":"ContainerStarted","Data":"9fd9379b894d4a657ab09b69f550ec3f6d2742ef4db3b6402795dcd48b416366"} Jul 21 16:53:43.423960 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:43.423895 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" podStartSLOduration=1.423863249 podStartE2EDuration="1.423863249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:53:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:53:43.420676058 +0000 UTC m=+1736.459511723" watchObservedRunningTime="2026-07-21 16:53:43.423863249 +0000 UTC m=+1736.462698912" Jul 21 16:53:43.748048 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:43.748013 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:43.749181 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:43.749157 2557 patch_prober.go:28] interesting pod/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 21 16:53:43.749292 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:43.749204 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 21 16:53:44.752072 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:44.752040 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:45.380578 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:45.380548 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:45.381618 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:45.381594 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:46.832914 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.832884 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:53:46.993882 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.993855 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-data\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994042 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.993888 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-credential-socket\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994042 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.993922 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-certs\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994042 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.993944 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-envoy\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994042 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994008 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-token\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994248 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994047 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-65244\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-kube-api-access-65244\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994248 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994082 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c564845-0c4d-49a3-b957-3b344a9b955e-istiod-ca-cert\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994248 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994148 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-socket\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994248 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994163 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:53:46.994248 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994189 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-podinfo\") pod \"2c564845-0c4d-49a3-b957-3b344a9b955e\" (UID: \"2c564845-0c4d-49a3-b957-3b344a9b955e\") " Jul 21 16:53:46.994248 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994205 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:53:46.994561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994491 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:53:46.994561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994532 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c564845-0c4d-49a3-b957-3b344a9b955e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:53:46.994561 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994542 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:46.994723 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994560 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:46.994813 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.994788 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:53:46.996339 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.996307 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 16:53:46.996457 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.996400 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:53:46.996457 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.996414 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:53:46.996552 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:46.996454 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-kube-api-access-65244" (OuterVolumeSpecName: "kube-api-access-65244") pod "2c564845-0c4d-49a3-b957-3b344a9b955e" (UID: "2c564845-0c4d-49a3-b957-3b344a9b955e"). InnerVolumeSpecName "kube-api-access-65244". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:53:47.095078 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.095043 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:47.095078 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.095072 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:47.095078 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.095083 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:47.095307 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.095091 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:47.095307 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.095098 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:47.095307 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.095106 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65244\" (UniqueName: \"kubernetes.io/projected/2c564845-0c4d-49a3-b957-3b344a9b955e-kube-api-access-65244\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:47.095307 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.095115 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c564845-0c4d-49a3-b957-3b344a9b955e-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:53:47.398466 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.398377 2557 generic.go:356] "Generic (PLEG): container finished" podID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerID="90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10" exitCode=0 Jul 21 16:53:47.398466 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.398431 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" Jul 21 16:53:47.398466 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.398456 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" event={"ID":"2c564845-0c4d-49a3-b957-3b344a9b955e","Type":"ContainerDied","Data":"90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10"} Jul 21 16:53:47.398805 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.398529 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n" event={"ID":"2c564845-0c4d-49a3-b957-3b344a9b955e","Type":"ContainerDied","Data":"900dd9df1d8c430cf8870861451423940646533c08b0aece90b015eebe18205e"} Jul 21 16:53:47.398805 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.398551 2557 scope.go:117] "RemoveContainer" containerID="90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10" Jul 21 16:53:47.408640 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.408622 2557 scope.go:117] "RemoveContainer" containerID="95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433" Jul 21 16:53:47.416654 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.416636 2557 scope.go:117] "RemoveContainer" containerID="90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10" Jul 21 16:53:47.416923 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:53:47.416903 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10\": container with ID starting with 90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10 not found: ID does not exist" containerID="90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10" Jul 21 16:53:47.416983 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.416931 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10"} err="failed to get container status \"90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10\": rpc error: code = NotFound desc = could not find container \"90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10\": container with ID starting with 90a9ff0af43788911e1a22f58b36f0a301c672fce8e639eefd353156f1414b10 not found: ID does not exist" Jul 21 16:53:47.416983 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.416948 2557 scope.go:117] "RemoveContainer" containerID="95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433" Jul 21 16:53:47.417188 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:53:47.417170 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433\": container with ID starting with 95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433 not found: ID does not exist" containerID="95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433" Jul 21 16:53:47.417233 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.417193 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433"} err="failed to get container status \"95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433\": rpc error: code = NotFound desc = could not find container \"95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433\": container with ID starting with 95699ff635dd21dab0e1750f0164369d87b51526ba1ac2d017afa9200105d433 not found: ID does not exist" Jul 21 16:53:47.420705 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.420686 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n"] Jul 21 16:53:47.425157 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.425137 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6ccfd18f-openshift-default-54fc78b57-whr7n"] Jul 21 16:53:47.489221 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:47.489189 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" path="/var/lib/kubelet/pods/2c564845-0c4d-49a3-b957-3b344a9b955e/volumes" Jul 21 16:53:57.461014 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:57.460984 2557 generic.go:356] "Generic (PLEG): container finished" podID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerID="500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419" exitCode=0 Jul 21 16:53:57.461365 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:57.461024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" event={"ID":"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c","Type":"ContainerDied","Data":"500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419"} Jul 21 16:53:57.461431 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:57.461405 2557 scope.go:117] "RemoveContainer" containerID="500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419" Jul 21 16:53:57.462485 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:57.462452 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:57.462553 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:57.462538 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:57.462590 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:57.462573 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:53:58.468309 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:58.468270 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" event={"ID":"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c","Type":"ContainerStarted","Data":"c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744"} Jul 21 16:53:58.749881 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:58.749837 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:53:58.750102 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:58.750029 2557 patch_prober.go:28] interesting pod/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 21 16:53:58.750102 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:58.750075 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 21 16:53:59.749209 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:59.749172 2557 patch_prober.go:28] interesting pod/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 21 16:53:59.750232 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:53:59.749801 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 21 16:54:00.380470 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:00.380431 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:54:00.748894 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:00.748858 2557 patch_prober.go:28] interesting pod/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 21 16:54:00.749220 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:00.749181 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 21 16:54:01.748776 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:01.748739 2557 patch_prober.go:28] interesting pod/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 21 16:54:01.749386 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:01.749356 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 21 16:54:02.752116 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:02.752081 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:54:02.753126 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:02.753105 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:54:24.895612 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:24.895576 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp"] Jul 21 16:54:24.896271 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:24.895846 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" containerID="cri-o://c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744" gracePeriod=30 Jul 21 16:54:30.155074 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.155010 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:54:30.300518 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300479 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-data\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300702 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300570 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istiod-ca-cert\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300702 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300593 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-podinfo\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300702 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300634 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-certs\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300702 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300665 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-envoy\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300702 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300688 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxpzf\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-kube-api-access-qxpzf\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300995 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300711 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-token\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300995 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300759 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-credential-socket\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300995 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300807 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-socket\") pod \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\" (UID: \"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c\") " Jul 21 16:54:30.300995 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.300958 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:54:30.301219 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.301056 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:54:30.301279 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.301233 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.301279 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.301252 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.301385 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.301323 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:54:30.301385 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.301335 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:54:30.301510 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.301385 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:54:30.302974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.302939 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 16:54:30.302974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.302957 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-kube-api-access-qxpzf" (OuterVolumeSpecName: "kube-api-access-qxpzf") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "kube-api-access-qxpzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:54:30.302974 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.302963 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:54:30.303148 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.303033 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" (UID: "8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:54:30.401730 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.401694 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.401730 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.401725 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.401730 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.401736 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.402004 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.401745 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.402004 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.401753 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxpzf\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-kube-api-access-qxpzf\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.402004 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.401761 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.402004 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.401772 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:54:30.642182 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.642147 2557 generic.go:356] "Generic (PLEG): container finished" podID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerID="c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744" exitCode=0 Jul 21 16:54:30.642362 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.642192 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" Jul 21 16:54:30.642362 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.642230 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" event={"ID":"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c","Type":"ContainerDied","Data":"c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744"} Jul 21 16:54:30.642362 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.642277 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp" event={"ID":"8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c","Type":"ContainerDied","Data":"9fd9379b894d4a657ab09b69f550ec3f6d2742ef4db3b6402795dcd48b416366"} Jul 21 16:54:30.642362 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.642299 2557 scope.go:117] "RemoveContainer" containerID="c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744" Jul 21 16:54:30.651738 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.651720 2557 scope.go:117] "RemoveContainer" containerID="500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419" Jul 21 16:54:30.660381 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.660362 2557 scope.go:117] "RemoveContainer" containerID="c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744" Jul 21 16:54:30.660642 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:54:30.660620 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744\": container with ID starting with c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744 not found: ID does not exist" containerID="c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744" Jul 21 16:54:30.660693 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.660651 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744"} err="failed to get container status \"c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744\": rpc error: code = NotFound desc = could not find container \"c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744\": container with ID starting with c1c7bcae758d3b8ecd3f1fc16bb7b07e02f50ba9af4ef85a618075d14b6d7744 not found: ID does not exist" Jul 21 16:54:30.660693 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.660669 2557 scope.go:117] "RemoveContainer" containerID="500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419" Jul 21 16:54:30.660944 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:54:30.660925 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419\": container with ID starting with 500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419 not found: ID does not exist" containerID="500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419" Jul 21 16:54:30.661014 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.660955 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419"} err="failed to get container status \"500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419\": rpc error: code = NotFound desc = could not find container \"500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419\": container with ID starting with 500c48941c164104ff2b5350b663f64d3f414470d30d74408946ea9237d50419 not found: ID does not exist" Jul 21 16:54:30.665923 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.665900 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp"] Jul 21 16:54:30.670463 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:30.670443 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9f938885-openshift-default-5d49c648bc-svffp"] Jul 21 16:54:31.489194 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:31.489163 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" path="/var/lib/kubelet/pods/8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c/volumes" Jul 21 16:54:47.528050 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:47.528011 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:54:47.535069 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:54:47.535045 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:56:08.036634 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.036597 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c"] Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037098 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037113 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037127 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037134 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037139 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037145 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037157 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" Jul 21 16:56:08.037166 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037162 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" Jul 21 16:56:08.037458 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037226 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" Jul 21 16:56:08.037458 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037234 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" Jul 21 16:56:08.037458 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.037242 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc3437f-5d5f-4c83-a46d-c42c3a07ae9c" containerName="istio-proxy" Jul 21 16:56:08.040279 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.040260 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.043704 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.043684 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-afe3d8e0-openshift-default-dockercfg-nxfqx\"" Jul 21 16:56:08.071089 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.071050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c"] Jul 21 16:56:08.074290 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074258 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-token\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074445 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074310 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-socket\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074445 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074398 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istiod-ca-cert\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074532 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074444 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-data\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074532 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074471 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxmvf\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-kube-api-access-vxmvf\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074532 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074494 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-envoy\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074663 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074573 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-certs\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074663 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074597 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-credential-socket\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.074663 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.074614 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-podinfo\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.175998 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.175964 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-token\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.175998 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176001 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-socket\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istiod-ca-cert\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-data\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176100 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxmvf\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-kube-api-access-vxmvf\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176125 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-envoy\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176184 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-certs\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-credential-socket\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176238 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176239 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-podinfo\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176602 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176428 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-socket\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176602 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176507 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-data\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176716 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176615 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-certs\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176716 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-credential-socket\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.176968 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.176949 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istiod-ca-cert\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.178563 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.178543 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-envoy\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.178644 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.178614 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-podinfo\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.183892 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.183874 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-token\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.184080 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.184064 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxmvf\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-kube-api-access-vxmvf\") pod \"e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.351375 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.351296 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:08.500094 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.500070 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c"] Jul 21 16:56:08.502085 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:56:08.502048 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad1f8ca_69f5_4104_8ada_9eae71d8f0a0.slice/crio-ea04e5d9ae5f0c0d312ef55788e011fffc06892b5068d86c8ec700c387f680e2 WatchSource:0}: Error finding container ea04e5d9ae5f0c0d312ef55788e011fffc06892b5068d86c8ec700c387f680e2: Status 404 returned error can't find the container with id ea04e5d9ae5f0c0d312ef55788e011fffc06892b5068d86c8ec700c387f680e2 Jul 21 16:56:08.504856 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.504806 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:56:08.504977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.504888 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:56:08.504977 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:08.504916 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:56:09.170838 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:09.170786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" event={"ID":"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0","Type":"ContainerStarted","Data":"5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f"} Jul 21 16:56:09.171252 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:09.170892 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" event={"ID":"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0","Type":"ContainerStarted","Data":"ea04e5d9ae5f0c0d312ef55788e011fffc06892b5068d86c8ec700c387f680e2"} Jul 21 16:56:09.207234 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:09.207160 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" podStartSLOduration=1.207145658 podStartE2EDuration="1.207145658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:56:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:56:09.204319516 +0000 UTC m=+1882.243155179" watchObservedRunningTime="2026-07-21 16:56:09.207145658 +0000 UTC m=+1882.245981319" Jul 21 16:56:09.351901 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:09.351863 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:09.356910 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:09.356885 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:10.175510 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:10.175424 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:10.176437 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:10.176417 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:21.240535 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:21.240494 2557 generic.go:356] "Generic (PLEG): container finished" podID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerID="5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f" exitCode=0 Jul 21 16:56:21.241193 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:21.240567 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" event={"ID":"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0","Type":"ContainerDied","Data":"5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f"} Jul 21 16:56:21.241193 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:21.240993 2557 scope.go:117] "RemoveContainer" containerID="5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f" Jul 21 16:56:21.242044 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:21.242020 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:56:21.242122 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:21.242092 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:56:21.242158 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:21.242126 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:56:22.251708 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:22.251669 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" event={"ID":"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0","Type":"ContainerStarted","Data":"2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772"} Jul 21 16:56:22.352358 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:22.352323 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:22.353594 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:22.353558 2557 patch_prober.go:28] interesting pod/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 16:56:22.353749 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:22.353625 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 16:56:23.352217 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:23.352171 2557 patch_prober.go:28] interesting pod/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 16:56:23.353991 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:23.353945 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 16:56:24.352331 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:24.352290 2557 patch_prober.go:28] interesting pod/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 16:56:24.353001 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:24.352971 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 16:56:25.175678 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:25.175627 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:25.352237 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:25.352191 2557 patch_prober.go:28] interesting pod/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 16:56:25.352578 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:25.352546 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 16:56:26.352022 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:26.351973 2557 patch_prober.go:28] interesting pod/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 21 16:56:26.352213 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:26.352058 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 21 16:56:27.355538 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:27.355503 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:27.356720 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:27.356696 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:44.752106 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:44.752072 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c"] Jul 21 16:56:44.752728 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:44.752342 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" containerID="cri-o://2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772" gracePeriod=30 Jul 21 16:56:49.995564 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:49.995538 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:50.087834 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.087737 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istiod-ca-cert\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.087834 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.087784 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-data\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.087854 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-token\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.087871 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxmvf\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-kube-api-access-vxmvf\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.087911 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-socket\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.087934 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-podinfo\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088009 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-credential-socket\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088061 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088038 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-certs\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088369 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088068 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-envoy\") pod \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\" (UID: \"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0\") " Jul 21 16:56:50.088369 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088181 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:56:50.088369 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088341 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:56:50.088537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088519 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:56:50.088537 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088525 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.088719 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088544 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.088719 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088550 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:56:50.088719 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.088650 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:56:50.090164 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.090137 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:56:50.090349 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.090322 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 16:56:50.090508 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.090481 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-kube-api-access-vxmvf" (OuterVolumeSpecName: "kube-api-access-vxmvf") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "kube-api-access-vxmvf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:56:50.090603 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.090558 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" (UID: "6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:56:50.189015 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.188981 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.189015 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.189009 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.189015 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.189018 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.189015 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.189025 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.189276 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.189033 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxmvf\" (UniqueName: \"kubernetes.io/projected/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-kube-api-access-vxmvf\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.189276 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.189042 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.189276 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.189051 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:56:50.411930 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.411843 2557 generic.go:356] "Generic (PLEG): container finished" podID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerID="2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772" exitCode=0 Jul 21 16:56:50.411930 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.411894 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" Jul 21 16:56:50.411930 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.411912 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" event={"ID":"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0","Type":"ContainerDied","Data":"2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772"} Jul 21 16:56:50.412156 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.411999 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c" event={"ID":"6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0","Type":"ContainerDied","Data":"ea04e5d9ae5f0c0d312ef55788e011fffc06892b5068d86c8ec700c387f680e2"} Jul 21 16:56:50.412156 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.412028 2557 scope.go:117] "RemoveContainer" containerID="2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772" Jul 21 16:56:50.421607 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.421590 2557 scope.go:117] "RemoveContainer" containerID="5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f" Jul 21 16:56:50.429502 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.429483 2557 scope.go:117] "RemoveContainer" containerID="2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772" Jul 21 16:56:50.429717 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:56:50.429700 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772\": container with ID starting with 2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772 not found: ID does not exist" containerID="2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772" Jul 21 16:56:50.429764 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.429723 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772"} err="failed to get container status \"2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772\": rpc error: code = NotFound desc = could not find container \"2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772\": container with ID starting with 2a55db13ff00913b052d42920dc795edd4ad14ed5c4ed6e850a0bc0d41ec2772 not found: ID does not exist" Jul 21 16:56:50.429764 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.429739 2557 scope.go:117] "RemoveContainer" containerID="5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f" Jul 21 16:56:50.430004 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:56:50.429984 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f\": container with ID starting with 5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f not found: ID does not exist" containerID="5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f" Jul 21 16:56:50.430076 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.430008 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f"} err="failed to get container status \"5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f\": rpc error: code = NotFound desc = could not find container \"5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f\": container with ID starting with 5a3bbe67a7474d8f97b21bcca3c878792e45b40a6b66ec42013bd66b07d46e8f not found: ID does not exist" Jul 21 16:56:50.440432 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.440408 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c"] Jul 21 16:56:50.446143 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:50.446122 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-afe3d8e0-openshift-default-7979c94d55-6bf2c"] Jul 21 16:56:51.488954 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:56:51.488923 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" path="/var/lib/kubelet/pods/6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0/volumes" Jul 21 16:57:06.931753 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.931718 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh"] Jul 21 16:57:06.932219 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.932178 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" Jul 21 16:57:06.932219 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.932191 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" Jul 21 16:57:06.932306 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.932266 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c564845-0c4d-49a3-b957-3b344a9b955e" containerName="istio-proxy" Jul 21 16:57:06.932306 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.932277 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" Jul 21 16:57:06.932306 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.932284 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" Jul 21 16:57:06.935395 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.935377 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:06.939024 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.939005 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e336b700-openshift-default-dockercfg-wnxsv\"" Jul 21 16:57:06.951894 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:06.951863 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh"] Jul 21 16:57:07.049535 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049505 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-certs\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049535 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049537 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-data\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049757 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049558 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgdtm\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-kube-api-access-zgdtm\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049757 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049645 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istiod-ca-cert\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049757 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049664 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-token\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049757 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049695 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-envoy\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049919 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049754 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-socket\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049919 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049859 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-credential-socket\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.049919 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.049888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-podinfo\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.150571 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-credential-socket\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.150760 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150585 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-podinfo\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.150760 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150631 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-certs\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.150760 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-data\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.150760 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150665 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgdtm\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-kube-api-access-zgdtm\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.150760 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150698 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istiod-ca-cert\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-token\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.150966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-envoy\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.151016 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-credential-socket\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151064 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.151021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-socket\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151272 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.151108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-data\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151272 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.151123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-certs\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151379 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.151349 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-socket\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.151476 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.151459 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istiod-ca-cert\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.153197 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.153175 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-podinfo\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.153304 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.153258 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-envoy\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.160316 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.160288 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-token\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.161136 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.161110 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgdtm\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-kube-api-access-zgdtm\") pod \"e2e-mt-e336b700-openshift-default-944d647bc-76grh\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.245651 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.245616 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:07.387523 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.387496 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh"] Jul 21 16:57:07.388933 ip-10-0-132-151 kubenswrapper[2557]: W0721 16:57:07.388903 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3f84a0_76d1_4d53_9e78_f8617a98c86a.slice/crio-9e62a16f74cb47e799a8cb102aab5ee6837447883a3731e3b05954172df0679a WatchSource:0}: Error finding container 9e62a16f74cb47e799a8cb102aab5ee6837447883a3731e3b05954172df0679a: Status 404 returned error can't find the container with id 9e62a16f74cb47e799a8cb102aab5ee6837447883a3731e3b05954172df0679a Jul 21 16:57:07.391182 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.391145 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:57:07.391300 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.391227 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:57:07.391300 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.391268 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:57:07.500944 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.500858 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" event={"ID":"8d3f84a0-76d1-4d53-9e78-f8617a98c86a","Type":"ContainerStarted","Data":"8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc"} Jul 21 16:57:07.500944 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.500915 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" event={"ID":"8d3f84a0-76d1-4d53-9e78-f8617a98c86a","Type":"ContainerStarted","Data":"9e62a16f74cb47e799a8cb102aab5ee6837447883a3731e3b05954172df0679a"} Jul 21 16:57:07.536450 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:07.536395 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" podStartSLOduration=1.536377892 podStartE2EDuration="1.536377892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 16:57:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 16:57:07.535140989 +0000 UTC m=+1940.573976677" watchObservedRunningTime="2026-07-21 16:57:07.536377892 +0000 UTC m=+1940.575213553" Jul 21 16:57:08.246011 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:08.245971 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:08.250698 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:08.250675 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:08.506757 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:08.506672 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:08.507770 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:08.507752 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:22.597492 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:22.597443 2557 generic.go:356] "Generic (PLEG): container finished" podID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerID="8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc" exitCode=0 Jul 21 16:57:22.597971 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:22.597486 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" event={"ID":"8d3f84a0-76d1-4d53-9e78-f8617a98c86a","Type":"ContainerDied","Data":"8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc"} Jul 21 16:57:22.597971 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:22.597910 2557 scope.go:117] "RemoveContainer" containerID="8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc" Jul 21 16:57:22.598994 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:22.598963 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:57:22.599131 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:22.599045 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:57:22.599131 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:22.599097 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 16:57:23.507521 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:23.507487 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:23.605272 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:23.605232 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" event={"ID":"8d3f84a0-76d1-4d53-9e78-f8617a98c86a","Type":"ContainerStarted","Data":"6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6"} Jul 21 16:57:24.245749 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:24.245703 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:24.247384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:24.247345 2557 patch_prober.go:28] interesting pod/e2e-mt-e336b700-openshift-default-944d647bc-76grh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 21 16:57:24.247533 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:24.247429 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 21 16:57:25.246520 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:25.246479 2557 patch_prober.go:28] interesting pod/e2e-mt-e336b700-openshift-default-944d647bc-76grh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 21 16:57:25.247182 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:25.247147 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 21 16:57:26.246341 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:26.246291 2557 patch_prober.go:28] interesting pod/e2e-mt-e336b700-openshift-default-944d647bc-76grh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 21 16:57:26.246556 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:26.246361 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 21 16:57:27.246440 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:27.246398 2557 patch_prober.go:28] interesting pod/e2e-mt-e336b700-openshift-default-944d647bc-76grh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 21 16:57:27.246739 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:27.246699 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 21 16:57:28.249688 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:28.249655 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:28.250114 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:28.249986 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:57:28.250714 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:57:28.250695 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:58:47.629332 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:58:47.629301 2557 scope.go:117] "RemoveContainer" containerID="2c6de9e3d5d5210c24842508d22b275cb610921694d20276f41fe7cf5b21e753" Jul 21 16:58:47.638869 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:58:47.638844 2557 scope.go:117] "RemoveContainer" containerID="3aab6829463c1da2ecd9a07c9d9a0a6e6bf445abfc36aec16616245c99fdf76e" Jul 21 16:59:19.720289 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:19.720253 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh"] Jul 21 16:59:19.720867 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:19.720458 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" containerID="cri-o://6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6" gracePeriod=30 Jul 21 16:59:24.984017 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:24.983954 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:59:25.142718 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.142682 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-socket\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.142923 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.142740 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-credential-socket\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.142923 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.142775 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zgdtm\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-kube-api-access-zgdtm\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.142923 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.142790 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-envoy\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.142923 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.142808 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istiod-ca-cert\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.143162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.142981 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-token\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.143162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:59:25.143162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143054 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-podinfo\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.143162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143095 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-data\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.143162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143127 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:59:25.143162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143159 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-certs\") pod \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\" (UID: \"8d3f84a0-76d1-4d53-9e78-f8617a98c86a\") " Jul 21 16:59:25.143499 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143421 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 16:59:25.143499 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143437 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:59:25.143622 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143498 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.143622 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143515 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.143622 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143529 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.144018 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.143995 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-data" (OuterVolumeSpecName: "istio-data") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:59:25.145283 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.145261 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 16:59:25.145389 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.145308 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-token" (OuterVolumeSpecName: "istio-token") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:59:25.145548 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.145529 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 16:59:25.145617 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.145545 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-kube-api-access-zgdtm" (OuterVolumeSpecName: "kube-api-access-zgdtm") pod "8d3f84a0-76d1-4d53-9e78-f8617a98c86a" (UID: "8d3f84a0-76d1-4d53-9e78-f8617a98c86a"). InnerVolumeSpecName "kube-api-access-zgdtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 16:59:25.244851 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.244737 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.244851 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.244764 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.244851 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.244773 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.244851 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.244786 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgdtm\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-kube-api-access-zgdtm\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.244851 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.244796 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.244851 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.244809 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d3f84a0-76d1-4d53-9e78-f8617a98c86a-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 16:59:25.249613 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.249578 2557 generic.go:356] "Generic (PLEG): container finished" podID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerID="6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6" exitCode=0 Jul 21 16:59:25.249750 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.249641 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" Jul 21 16:59:25.249750 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.249633 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" event={"ID":"8d3f84a0-76d1-4d53-9e78-f8617a98c86a","Type":"ContainerDied","Data":"6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6"} Jul 21 16:59:25.249841 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.249781 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh" event={"ID":"8d3f84a0-76d1-4d53-9e78-f8617a98c86a","Type":"ContainerDied","Data":"9e62a16f74cb47e799a8cb102aab5ee6837447883a3731e3b05954172df0679a"} Jul 21 16:59:25.249841 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.249804 2557 scope.go:117] "RemoveContainer" containerID="6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6" Jul 21 16:59:25.259290 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.259267 2557 scope.go:117] "RemoveContainer" containerID="8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc" Jul 21 16:59:25.267652 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.267636 2557 scope.go:117] "RemoveContainer" containerID="6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6" Jul 21 16:59:25.267926 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:59:25.267908 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6\": container with ID starting with 6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6 not found: ID does not exist" containerID="6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6" Jul 21 16:59:25.268009 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.267935 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6"} err="failed to get container status \"6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6\": rpc error: code = NotFound desc = could not find container \"6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6\": container with ID starting with 6f1c380d5e0ced3dd851b57d729d6de28b241c1a71c3fc0b202bc7fa4518fdd6 not found: ID does not exist" Jul 21 16:59:25.268009 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.267952 2557 scope.go:117] "RemoveContainer" containerID="8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc" Jul 21 16:59:25.268203 ip-10-0-132-151 kubenswrapper[2557]: E0721 16:59:25.268185 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc\": container with ID starting with 8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc not found: ID does not exist" containerID="8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc" Jul 21 16:59:25.268244 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.268207 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc"} err="failed to get container status \"8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc\": rpc error: code = NotFound desc = could not find container \"8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc\": container with ID starting with 8a33634179bf16f54c329633f1e3aa07b462884681676fd92e078dbbb4d474dc not found: ID does not exist" Jul 21 16:59:25.284011 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.283983 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh"] Jul 21 16:59:25.293942 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.293917 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e336b700-openshift-default-944d647bc-76grh"] Jul 21 16:59:25.491162 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:25.491131 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" path="/var/lib/kubelet/pods/8d3f84a0-76d1-4d53-9e78-f8617a98c86a/volumes" Jul 21 16:59:47.566469 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:47.566432 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:59:47.570480 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:47.570452 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 16:59:47.665384 ip-10-0-132-151 kubenswrapper[2557]: I0721 16:59:47.665345 2557 scope.go:117] "RemoveContainer" containerID="fdb49be2b5083a150a3e773fd5d1dfb899d5ad5bf422977e29cc50fbc4211d37" Jul 21 17:00:00.145450 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.145417 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r"] Jul 21 17:00:00.146002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.145872 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" Jul 21 17:00:00.146002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.145889 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad1f8ca-69f5-4104-8ada-9eae71d8f0a0" containerName="istio-proxy" Jul 21 17:00:00.146002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.145909 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" Jul 21 17:00:00.146002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.145915 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" Jul 21 17:00:00.146002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.145929 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" Jul 21 17:00:00.146002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.145935 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" Jul 21 17:00:00.146002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.146005 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" Jul 21 17:00:00.146232 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.146017 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d3f84a0-76d1-4d53-9e78-f8617a98c86a" containerName="istio-proxy" Jul 21 17:00:00.149087 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.149070 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r" Jul 21 17:00:00.152082 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.152059 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zwrvn\"" Jul 21 17:00:00.157025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.156996 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r"] Jul 21 17:00:00.161514 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.161492 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r" Jul 21 17:00:00.298291 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.298268 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r"] Jul 21 17:00:00.300486 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:00:00.300457 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7c3bb0_edf6_41cb_862a_e143e92b45f4.slice/crio-a036d767d8af2a132f41bd49f08a42e98138cb3d226b6d7a506f10922583b52d WatchSource:0}: Error finding container a036d767d8af2a132f41bd49f08a42e98138cb3d226b6d7a506f10922583b52d: Status 404 returned error can't find the container with id a036d767d8af2a132f41bd49f08a42e98138cb3d226b6d7a506f10922583b52d Jul 21 17:00:00.439849 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:00.439733 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r" event={"ID":"5f7c3bb0-edf6-41cb-862a-e143e92b45f4","Type":"ContainerStarted","Data":"a036d767d8af2a132f41bd49f08a42e98138cb3d226b6d7a506f10922583b52d"} Jul 21 17:00:01.446433 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:01.446401 2557 generic.go:356] "Generic (PLEG): container finished" podID="5f7c3bb0-edf6-41cb-862a-e143e92b45f4" containerID="116b3b759d3512f9e479b57d258781fb075333d5c0cbba72eaacea3f73d6374d" exitCode=0 Jul 21 17:00:01.446859 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:01.446494 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r" event={"ID":"5f7c3bb0-edf6-41cb-862a-e143e92b45f4","Type":"ContainerDied","Data":"116b3b759d3512f9e479b57d258781fb075333d5c0cbba72eaacea3f73d6374d"} Jul 21 17:00:02.587444 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:02.587416 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r" Jul 21 17:00:03.457374 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:03.457347 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r" Jul 21 17:00:03.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:03.457381 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744220-j5k8r" event={"ID":"5f7c3bb0-edf6-41cb-862a-e143e92b45f4","Type":"ContainerDied","Data":"a036d767d8af2a132f41bd49f08a42e98138cb3d226b6d7a506f10922583b52d"} Jul 21 17:00:03.457559 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:03.457433 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a036d767d8af2a132f41bd49f08a42e98138cb3d226b6d7a506f10922583b52d" Jul 21 17:00:19.150636 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.150586 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq"] Jul 21 17:00:19.151070 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.151055 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f7c3bb0-edf6-41cb-862a-e143e92b45f4" containerName="cleanup" Jul 21 17:00:19.151070 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.151071 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f7c3bb0-edf6-41cb-862a-e143e92b45f4" containerName="cleanup" Jul 21 17:00:19.151183 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.151164 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f7c3bb0-edf6-41cb-862a-e143e92b45f4" containerName="cleanup" Jul 21 17:00:19.154434 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.154414 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.159106 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.159086 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4a9a8c-openshift-default-dockercfg-grmr4\"" Jul 21 17:00:19.167158 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.167138 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq"] Jul 21 17:00:19.324878 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.324842 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-data\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.324878 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.324883 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-envoy\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.325109 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.324918 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-certs\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.325109 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.324951 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a65afcb-2471-46a3-8282-379f79237971-istio-podinfo\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.325109 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.324979 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a65afcb-2471-46a3-8282-379f79237971-istiod-ca-cert\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.325109 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.325002 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-istio-token\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.325109 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.325025 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj8st\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-kube-api-access-hj8st\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.325109 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.325067 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-credential-socket\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.325362 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.325150 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-socket\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.426734 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-envoy\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.426734 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426694 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-certs\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.426734 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426713 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a65afcb-2471-46a3-8282-379f79237971-istio-podinfo\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427098 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426748 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a65afcb-2471-46a3-8282-379f79237971-istiod-ca-cert\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427098 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426770 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-istio-token\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427098 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426802 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj8st\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-kube-api-access-hj8st\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427098 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-credential-socket\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427098 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426933 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-socket\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427098 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.426991 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-data\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427378 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.427212 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-certs\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427378 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.427347 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-data\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427459 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.427390 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-credential-socket\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427459 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.427413 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-socket\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.427627 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.427590 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a65afcb-2471-46a3-8282-379f79237971-istiod-ca-cert\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.429071 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.429043 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-envoy\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.429521 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.429498 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a65afcb-2471-46a3-8282-379f79237971-istio-podinfo\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.437175 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.437148 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-istio-token\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.437286 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.437189 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj8st\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-kube-api-access-hj8st\") pod \"e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.464746 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.464714 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:19.611204 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.611180 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq"] Jul 21 17:00:19.613513 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:00:19.613484 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a65afcb_2471_46a3_8282_379f79237971.slice/crio-b5f67e44b40973eda611fa996216efb28a9444cc8abc817f565428dc62f15a48 WatchSource:0}: Error finding container b5f67e44b40973eda611fa996216efb28a9444cc8abc817f565428dc62f15a48: Status 404 returned error can't find the container with id b5f67e44b40973eda611fa996216efb28a9444cc8abc817f565428dc62f15a48 Jul 21 17:00:19.615707 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.615673 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:19.615813 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.615749 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:19.615813 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:19.615795 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:20.549144 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:20.549102 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" event={"ID":"8a65afcb-2471-46a3-8282-379f79237971","Type":"ContainerStarted","Data":"a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7"} Jul 21 17:00:20.549565 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:20.549162 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" event={"ID":"8a65afcb-2471-46a3-8282-379f79237971","Type":"ContainerStarted","Data":"b5f67e44b40973eda611fa996216efb28a9444cc8abc817f565428dc62f15a48"} Jul 21 17:00:20.575986 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:20.575934 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" podStartSLOduration=1.5759164989999999 podStartE2EDuration="1.575916499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:00:20.573610594 +0000 UTC m=+2133.612446257" watchObservedRunningTime="2026-07-21 17:00:20.575916499 +0000 UTC m=+2133.614752160" Jul 21 17:00:21.464926 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:21.464886 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:21.469617 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:21.469588 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:21.554337 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:21.554303 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:21.555193 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:21.555174 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:27.601760 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.601726 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd"] Jul 21 17:00:27.606905 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.606881 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.611521 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.611486 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1a8da1-openshift-default-dockercfg-wskd6\"" Jul 21 17:00:27.616435 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.616411 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd"] Jul 21 17:00:27.705875 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.705840 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-token\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.705875 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.705882 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l28jw\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-kube-api-access-l28jw\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.706123 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.705941 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-data\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.706123 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.705957 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66f4bec-061c-45ec-b813-4051c5605cb3-istiod-ca-cert\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.706123 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.705972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-certs\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.706123 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.706056 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-envoy\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.706123 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.706090 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-podinfo\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.706123 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.706115 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-socket\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.706372 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.706160 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-credential-socket\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.806690 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.806642 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-credential-socket\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.806933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.806729 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-token\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.806933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.806752 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l28jw\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-kube-api-access-l28jw\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.806933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.806802 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-data\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.806933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.806844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66f4bec-061c-45ec-b813-4051c5605cb3-istiod-ca-cert\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.806933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.806868 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-certs\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.806933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.806901 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-envoy\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.807259 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.807045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-podinfo\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.807259 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.807082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-socket\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.807259 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.807128 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-credential-socket\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.807482 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.807435 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-certs\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.807482 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.807454 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-data\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.807577 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.807533 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66f4bec-061c-45ec-b813-4051c5605cb3-istiod-ca-cert\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.807614 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.807593 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-socket\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.809341 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.809314 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-envoy\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.809565 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.809548 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-podinfo\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.815528 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.815500 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-token\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.815716 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.815694 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l28jw\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-kube-api-access-l28jw\") pod \"e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:27.923960 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:27.923873 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:28.098436 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.094176 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd"] Jul 21 17:00:28.098635 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:00:28.098590 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66f4bec_061c_45ec_b813_4051c5605cb3.slice/crio-e331556e2155d1700d798b2cf7fa2d940a90156be9c380f9a928497f80e1fddb WatchSource:0}: Error finding container e331556e2155d1700d798b2cf7fa2d940a90156be9c380f9a928497f80e1fddb: Status 404 returned error can't find the container with id e331556e2155d1700d798b2cf7fa2d940a90156be9c380f9a928497f80e1fddb Jul 21 17:00:28.102207 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.102159 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:28.102435 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.102397 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:28.102561 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.102473 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:28.612662 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.612620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" event={"ID":"a66f4bec-061c-45ec-b813-4051c5605cb3","Type":"ContainerStarted","Data":"8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c"} Jul 21 17:00:28.613167 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.612686 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" event={"ID":"a66f4bec-061c-45ec-b813-4051c5605cb3","Type":"ContainerStarted","Data":"e331556e2155d1700d798b2cf7fa2d940a90156be9c380f9a928497f80e1fddb"} Jul 21 17:00:28.636971 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.636918 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" podStartSLOduration=1.636901057 podStartE2EDuration="1.636901057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:00:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:00:28.634652646 +0000 UTC m=+2141.673488398" watchObservedRunningTime="2026-07-21 17:00:28.636901057 +0000 UTC m=+2141.675736719" Jul 21 17:00:28.924748 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:28.924671 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:29.933361 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:29.933329 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:30.624569 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:30.624537 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:30.625538 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:30.625510 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:36.556605 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.556567 2557 patch_prober.go:28] interesting pod/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 17:00:36.557071 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.556623 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 17:00:36.668172 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.668141 2557 generic.go:356] "Generic (PLEG): container finished" podID="8a65afcb-2471-46a3-8282-379f79237971" containerID="a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7" exitCode=0 Jul 21 17:00:36.668348 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.668219 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" event={"ID":"8a65afcb-2471-46a3-8282-379f79237971","Type":"ContainerDied","Data":"a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7"} Jul 21 17:00:36.668623 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.668609 2557 scope.go:117] "RemoveContainer" containerID="a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7" Jul 21 17:00:36.669650 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.669613 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:36.669766 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.669685 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:36.669766 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:36.669726 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:37.684439 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:37.684385 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" event={"ID":"8a65afcb-2471-46a3-8282-379f79237971","Type":"ContainerStarted","Data":"511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af"} Jul 21 17:00:38.466000 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:38.465611 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:38.466361 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:38.466323 2557 patch_prober.go:28] interesting pod/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 17:00:38.466555 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:38.466529 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 17:00:39.465503 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:39.465462 2557 patch_prober.go:28] interesting pod/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 17:00:39.465951 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:39.465528 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 17:00:40.466100 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:40.466061 2557 patch_prober.go:28] interesting pod/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 17:00:40.466602 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:40.466120 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 17:00:41.465129 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:41.465087 2557 patch_prober.go:28] interesting pod/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 21 17:00:41.465306 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:41.465149 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 21 17:00:42.469249 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.469175 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:42.469626 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.469422 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:42.470183 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.470163 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:00:42.713846 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.713737 2557 generic.go:356] "Generic (PLEG): container finished" podID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerID="8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c" exitCode=0 Jul 21 17:00:42.713846 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.713811 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" event={"ID":"a66f4bec-061c-45ec-b813-4051c5605cb3","Type":"ContainerDied","Data":"8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c"} Jul 21 17:00:42.714253 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.714238 2557 scope.go:117] "RemoveContainer" containerID="8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c" Jul 21 17:00:42.715280 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.715252 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:42.715352 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.715335 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:42.715390 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:42.715377 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:00:43.721744 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:43.721707 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" event={"ID":"a66f4bec-061c-45ec-b813-4051c5605cb3","Type":"ContainerStarted","Data":"c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989"} Jul 21 17:00:43.924871 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:43.924812 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:43.926174 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:43.926146 2557 patch_prober.go:28] interesting pod/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 21 17:00:43.926297 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:43.926209 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 21 17:00:44.924409 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:44.924374 2557 patch_prober.go:28] interesting pod/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 21 17:00:44.924810 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:44.924443 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 21 17:00:45.625351 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:45.625324 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:45.924396 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:45.924297 2557 patch_prober.go:28] interesting pod/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 21 17:00:45.924396 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:45.924362 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 21 17:00:46.924603 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:46.924560 2557 patch_prober.go:28] interesting pod/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 21 17:00:46.925054 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:46.924621 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 21 17:00:47.981568 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:47.981533 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:00:47.984275 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:00:47.984242 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:01:50.607370 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.607333 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc"] Jul 21 17:01:50.610062 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.610046 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" Jul 21 17:01:50.612939 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.612918 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zwrvn\"" Jul 21 17:01:50.613075 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.612956 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 17:01:50.619096 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.619075 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc"] Jul 21 17:01:50.730231 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.730184 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/395f0f32-e55e-4cad-9cc5-8797fe2915a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc\" (UID: \"395f0f32-e55e-4cad-9cc5-8797fe2915a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" Jul 21 17:01:50.831527 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.831486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/395f0f32-e55e-4cad-9cc5-8797fe2915a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc\" (UID: \"395f0f32-e55e-4cad-9cc5-8797fe2915a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" Jul 21 17:01:50.832132 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.832113 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/395f0f32-e55e-4cad-9cc5-8797fe2915a4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc\" (UID: \"395f0f32-e55e-4cad-9cc5-8797fe2915a4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" Jul 21 17:01:50.921355 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:50.921277 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" Jul 21 17:01:51.066684 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:51.066651 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc"] Jul 21 17:01:51.067675 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:01:51.067651 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395f0f32_e55e_4cad_9cc5_8797fe2915a4.slice/crio-34e40d52f2a5bf36b14b7dd4b0232756c650eaf7360636a5fab7e9b0a1037279 WatchSource:0}: Error finding container 34e40d52f2a5bf36b14b7dd4b0232756c650eaf7360636a5fab7e9b0a1037279: Status 404 returned error can't find the container with id 34e40d52f2a5bf36b14b7dd4b0232756c650eaf7360636a5fab7e9b0a1037279 Jul 21 17:01:51.100144 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:51.100109 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" event={"ID":"395f0f32-e55e-4cad-9cc5-8797fe2915a4","Type":"ContainerStarted","Data":"34e40d52f2a5bf36b14b7dd4b0232756c650eaf7360636a5fab7e9b0a1037279"} Jul 21 17:01:52.108062 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:52.107972 2557 generic.go:356] "Generic (PLEG): container finished" podID="395f0f32-e55e-4cad-9cc5-8797fe2915a4" containerID="6c8ef2b863428eca9515623d1e73d65a2e1dce7e026ac690c46be285ef58ccb9" exitCode=0 Jul 21 17:01:52.108062 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:52.108046 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" event={"ID":"395f0f32-e55e-4cad-9cc5-8797fe2915a4","Type":"ContainerDied","Data":"6c8ef2b863428eca9515623d1e73d65a2e1dce7e026ac690c46be285ef58ccb9"} Jul 21 17:01:53.257223 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:53.257194 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" Jul 21 17:01:53.356528 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:53.356498 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/395f0f32-e55e-4cad-9cc5-8797fe2915a4-maas-api-service-ca\") pod \"395f0f32-e55e-4cad-9cc5-8797fe2915a4\" (UID: \"395f0f32-e55e-4cad-9cc5-8797fe2915a4\") " Jul 21 17:01:53.356897 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:53.356870 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395f0f32-e55e-4cad-9cc5-8797fe2915a4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "395f0f32-e55e-4cad-9cc5-8797fe2915a4" (UID: "395f0f32-e55e-4cad-9cc5-8797fe2915a4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:01:53.457300 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:53.457207 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/395f0f32-e55e-4cad-9cc5-8797fe2915a4-maas-api-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:01:54.124343 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:54.124310 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" event={"ID":"395f0f32-e55e-4cad-9cc5-8797fe2915a4","Type":"ContainerDied","Data":"34e40d52f2a5bf36b14b7dd4b0232756c650eaf7360636a5fab7e9b0a1037279"} Jul 21 17:01:54.124523 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:54.124368 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34e40d52f2a5bf36b14b7dd4b0232756c650eaf7360636a5fab7e9b0a1037279" Jul 21 17:01:54.124523 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:01:54.124310 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc" Jul 21 17:02:05.716622 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:05.716581 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc"] Jul 21 17:02:05.719255 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:05.719232 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-4a9a8c-z26kc"] Jul 21 17:02:07.489917 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:07.489882 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="395f0f32-e55e-4cad-9cc5-8797fe2915a4" path="/var/lib/kubelet/pods/395f0f32-e55e-4cad-9cc5-8797fe2915a4/volumes" Jul 21 17:02:12.892097 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:12.892017 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq"] Jul 21 17:02:12.892522 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:12.892225 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" containerID="cri-o://511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af" gracePeriod=30 Jul 21 17:02:18.179917 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.179890 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:02:18.195203 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195175 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-certs\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195344 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195245 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-istio-token\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195344 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195264 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-socket\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195344 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195291 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-envoy\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195344 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195329 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a65afcb-2471-46a3-8282-379f79237971-istiod-ca-cert\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195580 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195348 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-credential-socket\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195580 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195376 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a65afcb-2471-46a3-8282-379f79237971-istio-podinfo\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195580 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195415 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-data\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195580 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195439 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj8st\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-kube-api-access-hj8st\") pod \"8a65afcb-2471-46a3-8282-379f79237971\" (UID: \"8a65afcb-2471-46a3-8282-379f79237971\") " Jul 21 17:02:18.195580 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195461 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:18.195580 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195516 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:18.195918 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195674 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:18.195918 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195775 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.195918 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195791 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.195918 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.195805 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.196148 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.196015 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a65afcb-2471-46a3-8282-379f79237971-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:02:18.196279 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.196252 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-data" (OuterVolumeSpecName: "istio-data") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:18.197801 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.197774 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a65afcb-2471-46a3-8282-379f79237971-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:02:18.198039 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.198018 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-istio-token" (OuterVolumeSpecName: "istio-token") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:02:18.198124 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.198033 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-kube-api-access-hj8st" (OuterVolumeSpecName: "kube-api-access-hj8st") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "kube-api-access-hj8st". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:02:18.198124 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.198065 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8a65afcb-2471-46a3-8282-379f79237971" (UID: "8a65afcb-2471-46a3-8282-379f79237971"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:18.250933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.249938 2557 generic.go:356] "Generic (PLEG): container finished" podID="8a65afcb-2471-46a3-8282-379f79237971" containerID="511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af" exitCode=0 Jul 21 17:02:18.250933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.250107 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" event={"ID":"8a65afcb-2471-46a3-8282-379f79237971","Type":"ContainerDied","Data":"511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af"} Jul 21 17:02:18.250933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.250159 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" event={"ID":"8a65afcb-2471-46a3-8282-379f79237971","Type":"ContainerDied","Data":"b5f67e44b40973eda611fa996216efb28a9444cc8abc817f565428dc62f15a48"} Jul 21 17:02:18.250933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.250187 2557 scope.go:117] "RemoveContainer" containerID="511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af" Jul 21 17:02:18.250933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.250355 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq" Jul 21 17:02:18.262706 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.261788 2557 scope.go:117] "RemoveContainer" containerID="a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7" Jul 21 17:02:18.272555 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.272534 2557 scope.go:117] "RemoveContainer" containerID="511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af" Jul 21 17:02:18.272866 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:02:18.272845 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af\": container with ID starting with 511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af not found: ID does not exist" containerID="511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af" Jul 21 17:02:18.272941 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.272874 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af"} err="failed to get container status \"511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af\": rpc error: code = NotFound desc = could not find container \"511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af\": container with ID starting with 511d13ebf5862d7b86c30aa303573f824d4e7e6b6bc17c25ebf4abb4892369af not found: ID does not exist" Jul 21 17:02:18.272941 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.272893 2557 scope.go:117] "RemoveContainer" containerID="a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7" Jul 21 17:02:18.273159 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:02:18.273141 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7\": container with ID starting with a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7 not found: ID does not exist" containerID="a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7" Jul 21 17:02:18.273202 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.273166 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7"} err="failed to get container status \"a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7\": rpc error: code = NotFound desc = could not find container \"a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7\": container with ID starting with a2ffad074ec640187ee1bde3742700802feeb0af44d60a3a667947e646355da7 not found: ID does not exist" Jul 21 17:02:18.279156 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.279130 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq"] Jul 21 17:02:18.282061 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.282037 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4a9a8c-openshift-default-78f976577-9jswq"] Jul 21 17:02:18.296296 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.296269 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.296296 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.296294 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.296296 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.296303 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a65afcb-2471-46a3-8282-379f79237971-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.296505 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.296311 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a65afcb-2471-46a3-8282-379f79237971-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.296505 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.296319 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a65afcb-2471-46a3-8282-379f79237971-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:18.296505 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:18.296328 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj8st\" (UniqueName: \"kubernetes.io/projected/8a65afcb-2471-46a3-8282-379f79237971-kube-api-access-hj8st\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:19.490000 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:19.489965 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a65afcb-2471-46a3-8282-379f79237971" path="/var/lib/kubelet/pods/8a65afcb-2471-46a3-8282-379f79237971/volumes" Jul 21 17:02:34.872227 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:34.872194 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd"] Jul 21 17:02:34.872813 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:34.872427 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" containerID="cri-o://c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989" gracePeriod=30 Jul 21 17:02:35.728515 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.728484 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b"] Jul 21 17:02:35.728934 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.728914 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" Jul 21 17:02:35.728934 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.728930 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" Jul 21 17:02:35.729142 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.728946 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395f0f32-e55e-4cad-9cc5-8797fe2915a4" containerName="revoke-keys" Jul 21 17:02:35.729142 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.728955 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="395f0f32-e55e-4cad-9cc5-8797fe2915a4" containerName="revoke-keys" Jul 21 17:02:35.729142 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.728965 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" Jul 21 17:02:35.729142 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.728970 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" Jul 21 17:02:35.729142 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.729047 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" Jul 21 17:02:35.729142 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.729055 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="395f0f32-e55e-4cad-9cc5-8797fe2915a4" containerName="revoke-keys" Jul 21 17:02:35.731482 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.731458 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.734916 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.734885 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-44c780-openshift-default-dockercfg-pzn9q\"" Jul 21 17:02:35.749476 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.749450 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b"] Jul 21 17:02:35.862179 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862134 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-envoy\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862383 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862193 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-token\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862383 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862257 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-certs\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862383 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862284 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istiod-ca-cert\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862575 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-podinfo\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862575 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862429 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-socket\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862575 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862513 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-credential-socket\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862575 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862567 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wkl6\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-kube-api-access-4wkl6\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.862761 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.862677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-data\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.963901 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.963867 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-envoy\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.963901 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.963903 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-token\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964513 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.963929 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-certs\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964513 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.963944 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istiod-ca-cert\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964513 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.963975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-podinfo\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964513 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.963992 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-socket\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964513 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-credential-socket\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964513 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wkl6\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-kube-api-access-4wkl6\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964513 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964208 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-data\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964940 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964515 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-credential-socket\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964940 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964550 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-socket\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964940 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964557 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-certs\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964940 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964575 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-data\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.964940 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.964839 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istiod-ca-cert\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.966397 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.966373 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-envoy\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.966592 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.966575 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-podinfo\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.977128 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.977099 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-token\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:35.977813 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:35.977794 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wkl6\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-kube-api-access-4wkl6\") pod \"e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:36.042992 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.042910 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:36.180914 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.180886 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b"] Jul 21 17:02:36.182451 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:02:36.182428 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf287bda3_c5af_4d61_a28c_f73fad09fd8a.slice/crio-b65e557552f76061d97fdff2ae990f26d0d03d24af9e00d91403ad5f56de9093 WatchSource:0}: Error finding container b65e557552f76061d97fdff2ae990f26d0d03d24af9e00d91403ad5f56de9093: Status 404 returned error can't find the container with id b65e557552f76061d97fdff2ae990f26d0d03d24af9e00d91403ad5f56de9093 Jul 21 17:02:36.184510 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.184480 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:36.184603 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.184555 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:36.184664 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.184598 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:36.353507 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.353405 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" event={"ID":"f287bda3-c5af-4d61-a28c-f73fad09fd8a","Type":"ContainerStarted","Data":"8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940"} Jul 21 17:02:36.353507 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.353466 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" event={"ID":"f287bda3-c5af-4d61-a28c-f73fad09fd8a","Type":"ContainerStarted","Data":"b65e557552f76061d97fdff2ae990f26d0d03d24af9e00d91403ad5f56de9093"} Jul 21 17:02:36.375370 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:36.375319 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podStartSLOduration=1.37530143 podStartE2EDuration="1.37530143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:36.373770747 +0000 UTC m=+2269.412606410" watchObservedRunningTime="2026-07-21 17:02:36.37530143 +0000 UTC m=+2269.414137093" Jul 21 17:02:37.043202 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:37.043162 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:37.044692 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:37.044661 2557 patch_prober.go:28] interesting pod/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 17:02:37.044811 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:37.044715 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 17:02:38.048207 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:38.048176 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:38.364890 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:38.364784 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:38.365813 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:38.365795 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:40.133061 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.133003 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:02:40.206753 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.206716 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l28jw\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-kube-api-access-l28jw\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.206753 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.206757 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-credential-socket\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207029 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.206805 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66f4bec-061c-45ec-b813-4051c5605cb3-istiod-ca-cert\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207029 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.206875 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-podinfo\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207029 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.206923 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-data\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207029 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.206978 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-socket\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207029 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207008 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-envoy\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207287 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207058 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-token\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207287 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207096 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-certs\") pod \"a66f4bec-061c-45ec-b813-4051c5605cb3\" (UID: \"a66f4bec-061c-45ec-b813-4051c5605cb3\") " Jul 21 17:02:40.207287 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207140 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:40.207455 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207291 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66f4bec-061c-45ec-b813-4051c5605cb3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:02:40.207542 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207510 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:40.207542 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207521 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.207654 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207535 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:40.207654 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207558 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a66f4bec-061c-45ec-b813-4051c5605cb3-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.207790 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.207769 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-data" (OuterVolumeSpecName: "istio-data") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:40.209504 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.209479 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:02:40.209504 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.209490 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:02:40.209675 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.209616 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-kube-api-access-l28jw" (OuterVolumeSpecName: "kube-api-access-l28jw") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "kube-api-access-l28jw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:02:40.209675 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.209641 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-token" (OuterVolumeSpecName: "istio-token") pod "a66f4bec-061c-45ec-b813-4051c5605cb3" (UID: "a66f4bec-061c-45ec-b813-4051c5605cb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:02:40.308389 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.308353 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.308389 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.308385 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.308389 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.308394 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.308633 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.308404 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.308633 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.308412 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l28jw\" (UniqueName: \"kubernetes.io/projected/a66f4bec-061c-45ec-b813-4051c5605cb3-kube-api-access-l28jw\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.308633 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.308421 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.308633 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.308429 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a66f4bec-061c-45ec-b813-4051c5605cb3-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:02:40.377896 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.377861 2557 generic.go:356] "Generic (PLEG): container finished" podID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerID="c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989" exitCode=0 Jul 21 17:02:40.378070 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.377910 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" Jul 21 17:02:40.378070 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.377941 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" event={"ID":"a66f4bec-061c-45ec-b813-4051c5605cb3","Type":"ContainerDied","Data":"c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989"} Jul 21 17:02:40.378070 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.378004 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd" event={"ID":"a66f4bec-061c-45ec-b813-4051c5605cb3","Type":"ContainerDied","Data":"e331556e2155d1700d798b2cf7fa2d940a90156be9c380f9a928497f80e1fddb"} Jul 21 17:02:40.378070 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.378035 2557 scope.go:117] "RemoveContainer" containerID="c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989" Jul 21 17:02:40.391025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.391006 2557 scope.go:117] "RemoveContainer" containerID="8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c" Jul 21 17:02:40.399429 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.399412 2557 scope.go:117] "RemoveContainer" containerID="c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989" Jul 21 17:02:40.399667 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:02:40.399649 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989\": container with ID starting with c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989 not found: ID does not exist" containerID="c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989" Jul 21 17:02:40.399708 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.399679 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989"} err="failed to get container status \"c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989\": rpc error: code = NotFound desc = could not find container \"c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989\": container with ID starting with c9d11a37291d4359fb921241ec19797acbdf52248ee4574484fb7963dfb67989 not found: ID does not exist" Jul 21 17:02:40.399708 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.399698 2557 scope.go:117] "RemoveContainer" containerID="8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c" Jul 21 17:02:40.399928 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:02:40.399908 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c\": container with ID starting with 8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c not found: ID does not exist" containerID="8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c" Jul 21 17:02:40.399999 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.399932 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c"} err="failed to get container status \"8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c\": rpc error: code = NotFound desc = could not find container \"8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c\": container with ID starting with 8463ddc35ee99929560f39605c70c8b071b9733cc31fb60e6cc8d50f18158d3c not found: ID does not exist" Jul 21 17:02:40.405350 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.405326 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd"] Jul 21 17:02:40.409299 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:40.409278 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1a8da1-openshift-default-66bbb57947-fmshd"] Jul 21 17:02:41.488886 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:41.488848 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" path="/var/lib/kubelet/pods/a66f4bec-061c-45ec-b813-4051c5605cb3/volumes" Jul 21 17:02:43.726799 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.726762 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669"] Jul 21 17:02:43.727201 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.727185 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" Jul 21 17:02:43.727201 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.727198 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" Jul 21 17:02:43.727280 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.727207 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" Jul 21 17:02:43.727280 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.727213 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" Jul 21 17:02:43.727343 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.727289 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" Jul 21 17:02:43.727343 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.727296 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66f4bec-061c-45ec-b813-4051c5605cb3" containerName="istio-proxy" Jul 21 17:02:43.727343 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.727304 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a65afcb-2471-46a3-8282-379f79237971" containerName="istio-proxy" Jul 21 17:02:43.730490 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.730463 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.734326 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.734298 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a04cd4-openshift-default-dockercfg-j6mpw\"" Jul 21 17:02:43.746116 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.746091 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669"] Jul 21 17:02:43.842025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.841992 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-podinfo\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842220 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842050 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istiod-ca-cert\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842220 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842077 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-credential-socket\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842220 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842121 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-envoy\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842220 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842149 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppgl2\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-kube-api-access-ppgl2\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842220 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842184 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-socket\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842220 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842221 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-token\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842460 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842245 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-certs\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.842460 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.842314 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-data\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.942838 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.942792 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-token\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943037 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.942863 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-certs\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943037 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.942915 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-data\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943037 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.942961 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-podinfo\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943037 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.942983 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istiod-ca-cert\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943037 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943015 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-credential-socket\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943315 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-envoy\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943315 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943081 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppgl2\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-kube-api-access-ppgl2\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943315 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943117 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-socket\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943483 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943381 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-data\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943483 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943423 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-certs\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943597 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943493 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-socket\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943650 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943608 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-credential-socket\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.943755 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.943732 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istiod-ca-cert\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.945377 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.945350 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-envoy\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.945593 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.945577 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-podinfo\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.956059 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.956028 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-token\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:43.956916 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:43.956897 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppgl2\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-kube-api-access-ppgl2\") pod \"e2e-api-b-a04cd4-openshift-default-5496f89457-89669\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:44.042172 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.042089 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:44.185016 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.184986 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669"] Jul 21 17:02:44.186548 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:02:44.186523 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd1436f_7d4e_49bc_a1e1_4c33fce9931a.slice/crio-ba84b982c8798557edb230801c2b63d86fff358edfa6b6632396bcd68ff4bce3 WatchSource:0}: Error finding container ba84b982c8798557edb230801c2b63d86fff358edfa6b6632396bcd68ff4bce3: Status 404 returned error can't find the container with id ba84b982c8798557edb230801c2b63d86fff358edfa6b6632396bcd68ff4bce3 Jul 21 17:02:44.188563 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.188536 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:44.188623 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.188600 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:44.188661 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.188627 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:44.402763 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.402665 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" event={"ID":"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a","Type":"ContainerStarted","Data":"17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498"} Jul 21 17:02:44.402763 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.402727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" event={"ID":"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a","Type":"ContainerStarted","Data":"ba84b982c8798557edb230801c2b63d86fff358edfa6b6632396bcd68ff4bce3"} Jul 21 17:02:44.427075 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:44.427015 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podStartSLOduration=1.426993498 podStartE2EDuration="1.426993498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:02:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:02:44.425113759 +0000 UTC m=+2277.463949446" watchObservedRunningTime="2026-07-21 17:02:44.426993498 +0000 UTC m=+2277.465829161" Jul 21 17:02:45.042588 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:45.042554 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:45.047649 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:45.047622 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:45.408546 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:45.408471 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:45.409364 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:45.409341 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:02:54.412700 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:54.406961 2557 patch_prober.go:28] interesting pod/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 17:02:54.412700 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:54.407019 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 17:02:55.467502 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:55.467469 2557 generic.go:356] "Generic (PLEG): container finished" podID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerID="8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940" exitCode=0 Jul 21 17:02:55.467925 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:55.467541 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" event={"ID":"f287bda3-c5af-4d61-a28c-f73fad09fd8a","Type":"ContainerDied","Data":"8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940"} Jul 21 17:02:55.467995 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:55.467975 2557 scope.go:117] "RemoveContainer" containerID="8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940" Jul 21 17:02:55.468986 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:55.468959 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:55.469095 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:55.469036 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:55.469095 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:55.469079 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:02:56.479504 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:56.479467 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" event={"ID":"f287bda3-c5af-4d61-a28c-f73fad09fd8a","Type":"ContainerStarted","Data":"52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639"} Jul 21 17:02:57.044017 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:57.043971 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:02:57.044243 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:57.044112 2557 patch_prober.go:28] interesting pod/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 17:02:57.044243 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:57.044157 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 17:02:58.043659 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:58.043609 2557 patch_prober.go:28] interesting pod/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 17:02:58.044227 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:58.043685 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 17:02:59.044398 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:59.044344 2557 patch_prober.go:28] interesting pod/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 17:02:59.044988 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:02:59.044438 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 17:03:00.043988 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:00.043948 2557 patch_prober.go:28] interesting pod/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 21 17:03:00.044304 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:00.044262 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 21 17:03:01.046945 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:01.046911 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:03:01.047379 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:01.047229 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:03:01.048110 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:01.048090 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:03:01.450056 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:01.449956 2557 patch_prober.go:28] interesting pod/e2e-api-b-a04cd4-openshift-default-5496f89457-89669 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 17:03:01.450056 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:01.450038 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 17:03:03.518273 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:03.518243 2557 generic.go:356] "Generic (PLEG): container finished" podID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerID="17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498" exitCode=0 Jul 21 17:03:03.518715 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:03.518320 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" event={"ID":"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a","Type":"ContainerDied","Data":"17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498"} Jul 21 17:03:03.518765 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:03.518746 2557 scope.go:117] "RemoveContainer" containerID="17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498" Jul 21 17:03:03.519781 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:03.519752 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:03:03.519888 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:03.519849 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:03:03.519931 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:03.519890 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:03:04.525220 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:04.525188 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" event={"ID":"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a","Type":"ContainerStarted","Data":"2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47"} Jul 21 17:03:05.045382 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:05.045330 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:03:05.045663 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:05.045515 2557 patch_prober.go:28] interesting pod/e2e-api-b-a04cd4-openshift-default-5496f89457-89669 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 17:03:05.045663 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:05.045575 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 17:03:06.043263 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:06.043214 2557 patch_prober.go:28] interesting pod/e2e-api-b-a04cd4-openshift-default-5496f89457-89669 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 17:03:06.043782 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:06.043301 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 17:03:07.043366 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:07.043325 2557 patch_prober.go:28] interesting pod/e2e-api-b-a04cd4-openshift-default-5496f89457-89669 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 17:03:07.044046 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:07.044013 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 17:03:08.042793 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:08.042724 2557 patch_prober.go:28] interesting pod/e2e-api-b-a04cd4-openshift-default-5496f89457-89669 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 17:03:08.043037 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:08.042815 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 17:03:09.043525 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:09.043487 2557 patch_prober.go:28] interesting pod/e2e-api-b-a04cd4-openshift-default-5496f89457-89669 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 21 17:03:09.044149 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:09.044115 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 21 17:03:10.046195 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:10.046161 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:03:10.046675 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:10.046549 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:03:10.047098 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:10.047078 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:03:26.116380 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:26.116346 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b"] Jul 21 17:03:26.116958 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:26.116565 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" containerID="cri-o://52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639" gracePeriod=30 Jul 21 17:03:31.049674 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.049624 2557 patch_prober.go:28] interesting pod/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 21 17:03:31.050090 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.049710 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 17:03:31.369306 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.369277 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:03:31.511731 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.511701 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-certs\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.511938 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.511752 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-credential-socket\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.511938 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.511776 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-podinfo\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.511938 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.511801 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-token\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.511938 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.511917 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-data\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.512172 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.511976 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wkl6\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-kube-api-access-4wkl6\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.512172 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512031 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-socket\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.512172 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512087 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istiod-ca-cert\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.512172 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512130 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-envoy\") pod \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\" (UID: \"f287bda3-c5af-4d61-a28c-f73fad09fd8a\") " Jul 21 17:03:31.512392 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512057 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:31.512392 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512104 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:31.512392 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512248 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:31.512530 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512412 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:03:31.512530 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512493 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.512530 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512512 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.512530 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512527 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.512685 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.512540 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.513082 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.513051 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-data" (OuterVolumeSpecName: "istio-data") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:31.514148 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.514121 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-kube-api-access-4wkl6" (OuterVolumeSpecName: "kube-api-access-4wkl6") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "kube-api-access-4wkl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:03:31.514250 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.514185 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:31.514376 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.514361 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-token" (OuterVolumeSpecName: "istio-token") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:03:31.514459 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.514440 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f287bda3-c5af-4d61-a28c-f73fad09fd8a" (UID: "f287bda3-c5af-4d61-a28c-f73fad09fd8a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:03:31.614025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.613936 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.614025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.613967 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.614025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.613976 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.614025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.613988 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f287bda3-c5af-4d61-a28c-f73fad09fd8a-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.614025 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.613997 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wkl6\" (UniqueName: \"kubernetes.io/projected/f287bda3-c5af-4d61-a28c-f73fad09fd8a-kube-api-access-4wkl6\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:31.680700 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.680665 2557 generic.go:356] "Generic (PLEG): container finished" podID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerID="52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639" exitCode=0 Jul 21 17:03:31.680927 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.680715 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" Jul 21 17:03:31.680927 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.680721 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" event={"ID":"f287bda3-c5af-4d61-a28c-f73fad09fd8a","Type":"ContainerDied","Data":"52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639"} Jul 21 17:03:31.680927 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.680774 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b" event={"ID":"f287bda3-c5af-4d61-a28c-f73fad09fd8a","Type":"ContainerDied","Data":"b65e557552f76061d97fdff2ae990f26d0d03d24af9e00d91403ad5f56de9093"} Jul 21 17:03:31.680927 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.680801 2557 scope.go:117] "RemoveContainer" containerID="52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639" Jul 21 17:03:31.690742 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.690722 2557 scope.go:117] "RemoveContainer" containerID="8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940" Jul 21 17:03:31.699049 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.699029 2557 scope.go:117] "RemoveContainer" containerID="52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639" Jul 21 17:03:31.699397 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:03:31.699363 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639\": container with ID starting with 52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639 not found: ID does not exist" containerID="52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639" Jul 21 17:03:31.699482 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.699398 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639"} err="failed to get container status \"52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639\": rpc error: code = NotFound desc = could not find container \"52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639\": container with ID starting with 52a2ead671252412634edf2002a30328b5662d03594ef8c08bafb981f6b24639 not found: ID does not exist" Jul 21 17:03:31.699482 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.699421 2557 scope.go:117] "RemoveContainer" containerID="8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940" Jul 21 17:03:31.699707 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:03:31.699690 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940\": container with ID starting with 8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940 not found: ID does not exist" containerID="8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940" Jul 21 17:03:31.699762 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.699711 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940"} err="failed to get container status \"8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940\": rpc error: code = NotFound desc = could not find container \"8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940\": container with ID starting with 8ef665c5d4d1221ce2c92a3d041cbffffe8413a9d805fd19d9b293d446eee940 not found: ID does not exist" Jul 21 17:03:31.704257 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.704232 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b"] Jul 21 17:03:31.710227 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:31.710207 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-44c780-openshift-default-7fb44968c5-zcq4b"] Jul 21 17:03:33.489077 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:33.489046 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" path="/var/lib/kubelet/pods/f287bda3-c5af-4d61-a28c-f73fad09fd8a/volumes" Jul 21 17:03:53.761229 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:53.761200 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669"] Jul 21 17:03:53.761732 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:53.761419 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" containerID="cri-o://2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47" gracePeriod=30 Jul 21 17:03:54.714560 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.714526 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4"] Jul 21 17:03:54.714951 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.714938 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" Jul 21 17:03:54.714951 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.714952 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" Jul 21 17:03:54.715044 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.715030 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" Jul 21 17:03:54.715044 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.715039 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" Jul 21 17:03:54.718941 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.717977 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.721767 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.721743 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-07bb43-openshift-default-dockercfg-27nm9\"" Jul 21 17:03:54.731813 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.731786 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4"] Jul 21 17:03:54.812489 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812457 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-token\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812497 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb9dp\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-kube-api-access-mb9dp\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812525 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-credential-socket\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812589 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-certs\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812620 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-envoy\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812640 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-podinfo\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812665 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istiod-ca-cert\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812717 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-data\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.812910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.812748 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-socket\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.913847 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.913789 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-podinfo\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914034 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.913857 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istiod-ca-cert\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914034 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.913890 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-data\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914034 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.913922 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-socket\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914034 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.913975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-token\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914034 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914013 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb9dp\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-kube-api-access-mb9dp\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914330 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914053 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-credential-socket\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914330 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914098 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-certs\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914330 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-envoy\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914496 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914384 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-data\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914496 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914428 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-credential-socket\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914496 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914470 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-socket\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914658 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914610 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-certs\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.914717 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.914684 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istiod-ca-cert\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.916301 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.916279 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-envoy\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.916428 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.916411 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-podinfo\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.923256 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.923237 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb9dp\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-kube-api-access-mb9dp\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:54.923337 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:54.923261 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-token\") pod \"e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:55.032204 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.032167 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:55.049734 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.049705 2557 patch_prober.go:28] interesting pod/e2e-api-b-a04cd4-openshift-default-5496f89457-89669 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 17:03:55.049937 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.049772 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 17:03:55.179016 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.178992 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4"] Jul 21 17:03:55.181264 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:03:55.181237 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a6c52cb_eed2_4cd8_b29f_e31853037a9e.slice/crio-78718dba2ee43737389b7a45663fab367f02343e2b2bfe53c20758ac9a131f49 WatchSource:0}: Error finding container 78718dba2ee43737389b7a45663fab367f02343e2b2bfe53c20758ac9a131f49: Status 404 returned error can't find the container with id 78718dba2ee43737389b7a45663fab367f02343e2b2bfe53c20758ac9a131f49 Jul 21 17:03:55.183283 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.183253 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:03:55.183363 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.183314 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:03:55.183363 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.183341 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:03:55.807622 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.807585 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" event={"ID":"3a6c52cb-eed2-4cd8-b29f-e31853037a9e","Type":"ContainerStarted","Data":"f95ecda07d7e0d69814f0e40b01035df8e19f3c07ae5d0df957931d56e6f81a0"} Jul 21 17:03:55.807844 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.807650 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" event={"ID":"3a6c52cb-eed2-4cd8-b29f-e31853037a9e","Type":"ContainerStarted","Data":"78718dba2ee43737389b7a45663fab367f02343e2b2bfe53c20758ac9a131f49"} Jul 21 17:03:55.830704 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:55.830655 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" podStartSLOduration=1.830641273 podStartE2EDuration="1.830641273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:03:55.82769714 +0000 UTC m=+2348.866532805" watchObservedRunningTime="2026-07-21 17:03:55.830641273 +0000 UTC m=+2348.869476933" Jul 21 17:03:56.032953 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:56.032906 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:56.038240 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:56.038212 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:56.813768 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:56.813737 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:56.814523 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:56.814501 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:03:59.019146 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.019086 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:03:59.052800 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.052769 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-socket\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.052831 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-token\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.052864 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-data\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.052891 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istiod-ca-cert\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.052916 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-podinfo\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053002 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.052985 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-certs\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053290 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053015 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-envoy\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053290 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053041 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-credential-socket\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053290 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053070 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:59.053290 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053161 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppgl2\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-kube-api-access-ppgl2\") pod \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\" (UID: \"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a\") " Jul 21 17:03:59.053290 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053273 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:03:59.053565 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053549 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:59.053619 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053561 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:59.053674 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053651 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.053674 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053668 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.053781 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053682 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.053781 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053698 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.053781 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.053722 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-data" (OuterVolumeSpecName: "istio-data") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:59.055642 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.055601 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:03:59.055761 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.055679 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:03:59.055834 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.055770 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-kube-api-access-ppgl2" (OuterVolumeSpecName: "kube-api-access-ppgl2") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "kube-api-access-ppgl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:03:59.055834 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.055794 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-token" (OuterVolumeSpecName: "istio-token") pod "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" (UID: "9fd1436f-7d4e-49bc-a1e1-4c33fce9931a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:03:59.155163 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.155127 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.155163 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.155159 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppgl2\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-kube-api-access-ppgl2\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.155163 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.155169 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.155396 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.155177 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.155396 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.155184 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:03:59.831217 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.831131 2557 generic.go:356] "Generic (PLEG): container finished" podID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerID="2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47" exitCode=0 Jul 21 17:03:59.831217 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.831196 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" Jul 21 17:03:59.831416 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.831211 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" event={"ID":"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a","Type":"ContainerDied","Data":"2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47"} Jul 21 17:03:59.831416 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.831268 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669" event={"ID":"9fd1436f-7d4e-49bc-a1e1-4c33fce9931a","Type":"ContainerDied","Data":"ba84b982c8798557edb230801c2b63d86fff358edfa6b6632396bcd68ff4bce3"} Jul 21 17:03:59.831416 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.831292 2557 scope.go:117] "RemoveContainer" containerID="2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47" Jul 21 17:03:59.840147 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.840127 2557 scope.go:117] "RemoveContainer" containerID="17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498" Jul 21 17:03:59.848292 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.848269 2557 scope.go:117] "RemoveContainer" containerID="2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47" Jul 21 17:03:59.848620 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:03:59.848590 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47\": container with ID starting with 2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47 not found: ID does not exist" containerID="2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47" Jul 21 17:03:59.848858 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.848802 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47"} err="failed to get container status \"2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47\": rpc error: code = NotFound desc = could not find container \"2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47\": container with ID starting with 2f2c6307b9dc7d8c1622d12eb6b876cac74e26f8828904737cbb51b8cb8b5d47 not found: ID does not exist" Jul 21 17:03:59.848858 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.848860 2557 scope.go:117] "RemoveContainer" containerID="17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498" Jul 21 17:03:59.849135 ip-10-0-132-151 kubenswrapper[2557]: E0721 17:03:59.849118 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498\": container with ID starting with 17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498 not found: ID does not exist" containerID="17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498" Jul 21 17:03:59.849190 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.849149 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498"} err="failed to get container status \"17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498\": rpc error: code = NotFound desc = could not find container \"17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498\": container with ID starting with 17292489aa18a360eef8ce3770d10b874c7f79e39f9cb48ec892d2a017ea3498 not found: ID does not exist" Jul 21 17:03:59.851180 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.851160 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669"] Jul 21 17:03:59.854499 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:03:59.854475 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a04cd4-openshift-default-5496f89457-89669"] Jul 21 17:04:01.489924 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:01.489892 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" path="/var/lib/kubelet/pods/9fd1436f-7d4e-49bc-a1e1-4c33fce9931a/volumes" Jul 21 17:04:02.888210 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888171 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc"] Jul 21 17:04:02.888704 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888649 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" Jul 21 17:04:02.888704 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888663 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" Jul 21 17:04:02.888704 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888681 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" Jul 21 17:04:02.888704 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888690 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f287bda3-c5af-4d61-a28c-f73fad09fd8a" containerName="istio-proxy" Jul 21 17:04:02.888970 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888714 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" Jul 21 17:04:02.888970 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888724 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" Jul 21 17:04:02.888970 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888809 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" Jul 21 17:04:02.888970 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.888837 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd1436f-7d4e-49bc-a1e1-4c33fce9931a" containerName="istio-proxy" Jul 21 17:04:02.891739 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.891720 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.894616 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.894596 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-1e5534-openshift-default-dockercfg-mq7ln\"" Jul 21 17:04:02.902174 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.902151 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc"] Jul 21 17:04:02.989989 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.989951 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-data\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990193 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.989997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-istio-token\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990193 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.990085 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17651380-163e-4703-ae76-091c7f6b10f5-istio-podinfo\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990193 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.990145 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-socket\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990367 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.990206 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-credential-socket\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990367 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.990244 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-certs\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990367 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.990269 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-envoy\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990367 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.990301 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17651380-163e-4703-ae76-091c7f6b10f5-istiod-ca-cert\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:02.990367 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:02.990325 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6fv5\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-kube-api-access-h6fv5\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091366 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091327 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17651380-163e-4703-ae76-091c7f6b10f5-istio-podinfo\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091366 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-socket\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091629 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091425 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-credential-socket\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091629 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091452 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-certs\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091629 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-envoy\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091629 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091503 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17651380-163e-4703-ae76-091c7f6b10f5-istiod-ca-cert\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091869 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091636 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6fv5\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-kube-api-access-h6fv5\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091869 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091759 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-data\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.091869 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091785 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-istio-token\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.092046 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091932 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-credential-socket\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.092046 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.091965 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-certs\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.092161 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.092122 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-data\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.092228 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.092188 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-socket\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.092600 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.092579 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17651380-163e-4703-ae76-091c7f6b10f5-istiod-ca-cert\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.094277 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.094254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-envoy\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.094386 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.094372 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17651380-163e-4703-ae76-091c7f6b10f5-istio-podinfo\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.102706 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.102681 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-istio-token\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.102884 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.102866 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6fv5\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-kube-api-access-h6fv5\") pod \"e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.203567 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.203467 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:03.345574 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.345539 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc"] Jul 21 17:04:03.347012 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:04:03.346991 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17651380_163e_4703_ae76_091c7f6b10f5.slice/crio-3fa5ee2fcdce9b097f5407efee3783257454337cd7f44f943e9b5b7d9692e2ca WatchSource:0}: Error finding container 3fa5ee2fcdce9b097f5407efee3783257454337cd7f44f943e9b5b7d9692e2ca: Status 404 returned error can't find the container with id 3fa5ee2fcdce9b097f5407efee3783257454337cd7f44f943e9b5b7d9692e2ca Jul 21 17:04:03.349226 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.349199 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:03.349311 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.349258 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:03.349311 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.349292 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:03.853806 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.853764 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" event={"ID":"17651380-163e-4703-ae76-091c7f6b10f5","Type":"ContainerStarted","Data":"f91814674fa4e1ed9d5b33bed850e3f023736ebd2b9f96d7dd0ede5bc8870373"} Jul 21 17:04:03.854019 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.853853 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" event={"ID":"17651380-163e-4703-ae76-091c7f6b10f5","Type":"ContainerStarted","Data":"3fa5ee2fcdce9b097f5407efee3783257454337cd7f44f943e9b5b7d9692e2ca"} Jul 21 17:04:03.875610 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:03.875558 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" podStartSLOduration=1.8755418289999999 podStartE2EDuration="1.875541829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:04:03.874388093 +0000 UTC m=+2356.913223756" watchObservedRunningTime="2026-07-21 17:04:03.875541829 +0000 UTC m=+2356.914377490" Jul 21 17:04:04.204696 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:04.204606 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:04.209668 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:04.209642 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:04.859225 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:04.859194 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:04.860218 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:04.860201 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:12.856128 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:12.856063 2557 patch_prober.go:28] interesting pod/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 17:04:12.858297 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:12.856196 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 17:04:13.909049 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:13.909015 2557 generic.go:356] "Generic (PLEG): container finished" podID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerID="f95ecda07d7e0d69814f0e40b01035df8e19f3c07ae5d0df957931d56e6f81a0" exitCode=0 Jul 21 17:04:13.909440 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:13.909087 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" event={"ID":"3a6c52cb-eed2-4cd8-b29f-e31853037a9e","Type":"ContainerDied","Data":"f95ecda07d7e0d69814f0e40b01035df8e19f3c07ae5d0df957931d56e6f81a0"} Jul 21 17:04:13.909502 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:13.909455 2557 scope.go:117] "RemoveContainer" containerID="f95ecda07d7e0d69814f0e40b01035df8e19f3c07ae5d0df957931d56e6f81a0" Jul 21 17:04:13.910487 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:13.910459 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:13.910559 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:13.910543 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:13.910604 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:13.910582 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:14.916408 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:14.916369 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" event={"ID":"3a6c52cb-eed2-4cd8-b29f-e31853037a9e","Type":"ContainerStarted","Data":"9b5e224348869174e42bc8de1ed80600c4775ce95ddba8e729b2227899d08792"} Jul 21 17:04:15.033047 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:15.033008 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:04:15.033367 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:15.033335 2557 patch_prober.go:28] interesting pod/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 17:04:15.033472 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:15.033399 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 17:04:16.034804 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:16.034767 2557 patch_prober.go:28] interesting pod/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 17:04:16.035422 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:16.035389 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 17:04:17.033549 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:17.033499 2557 patch_prober.go:28] interesting pod/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 17:04:17.033877 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:17.033809 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 17:04:18.036860 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:18.033220 2557 patch_prober.go:28] interesting pod/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 21 17:04:18.036860 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:18.033286 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 21 17:04:19.037278 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.037246 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:04:19.037769 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.037625 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:04:19.038432 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.038415 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:04:19.861041 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.861005 2557 patch_prober.go:28] interesting pod/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 17:04:19.861254 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.861062 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 17:04:19.946526 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.946489 2557 generic.go:356] "Generic (PLEG): container finished" podID="17651380-163e-4703-ae76-091c7f6b10f5" containerID="f91814674fa4e1ed9d5b33bed850e3f023736ebd2b9f96d7dd0ede5bc8870373" exitCode=0 Jul 21 17:04:19.946694 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.946564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" event={"ID":"17651380-163e-4703-ae76-091c7f6b10f5","Type":"ContainerDied","Data":"f91814674fa4e1ed9d5b33bed850e3f023736ebd2b9f96d7dd0ede5bc8870373"} Jul 21 17:04:19.947157 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.947140 2557 scope.go:117] "RemoveContainer" containerID="f91814674fa4e1ed9d5b33bed850e3f023736ebd2b9f96d7dd0ede5bc8870373" Jul 21 17:04:19.948237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.948209 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:19.948305 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.948288 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:19.948355 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:19.948341 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:04:20.960116 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:20.960070 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" event={"ID":"17651380-163e-4703-ae76-091c7f6b10f5","Type":"ContainerStarted","Data":"0e5d83d0b35344ef84cd0971a0a729c0b4b20b4e397ed3d489091c0d1bcad634"} Jul 21 17:04:21.203855 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:21.203811 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:21.204091 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:21.204070 2557 patch_prober.go:28] interesting pod/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 17:04:21.204178 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:21.204125 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 17:04:22.204100 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:22.204055 2557 patch_prober.go:28] interesting pod/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 17:04:22.204495 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:22.204130 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 17:04:23.204728 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:23.204689 2557 patch_prober.go:28] interesting pod/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 17:04:23.205174 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:23.204749 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 17:04:24.204355 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:24.204316 2557 patch_prober.go:28] interesting pod/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 21 17:04:24.204526 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:24.204376 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 21 17:04:25.208415 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:25.208384 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:25.209007 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:25.208772 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:25.209792 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:25.209766 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:04:47.600119 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:47.600093 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 17:04:47.604058 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:04:47.604037 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 17:08:47.774354 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:08:47.774318 2557 scope.go:117] "RemoveContainer" containerID="6c8ef2b863428eca9515623d1e73d65a2e1dce7e026ac690c46be285ef58ccb9" Jul 21 17:09:47.650666 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:09:47.650585 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 17:09:47.656690 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:09:47.656661 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 17:11:37.179905 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:37.179873 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4"] Jul 21 17:11:37.180453 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:37.180098 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" containerID="cri-o://9b5e224348869174e42bc8de1ed80600c4775ce95ddba8e729b2227899d08792" gracePeriod=30 Jul 21 17:11:42.342328 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.342284 2557 generic.go:356] "Generic (PLEG): container finished" podID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerID="9b5e224348869174e42bc8de1ed80600c4775ce95ddba8e729b2227899d08792" exitCode=0 Jul 21 17:11:42.342713 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.342362 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" event={"ID":"3a6c52cb-eed2-4cd8-b29f-e31853037a9e","Type":"ContainerDied","Data":"9b5e224348869174e42bc8de1ed80600c4775ce95ddba8e729b2227899d08792"} Jul 21 17:11:42.342713 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.342424 2557 scope.go:117] "RemoveContainer" containerID="f95ecda07d7e0d69814f0e40b01035df8e19f3c07ae5d0df957931d56e6f81a0" Jul 21 17:11:42.435742 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.435717 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:11:42.513357 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513325 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-credential-socket\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513357 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513357 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-socket\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513608 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513388 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-data\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513608 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513413 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-envoy\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513608 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513440 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-certs\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513608 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513487 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-podinfo\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513608 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513515 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mb9dp\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-kube-api-access-mb9dp\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513608 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513540 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istiod-ca-cert\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.513608 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513601 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-token\") pod \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\" (UID: \"3a6c52cb-eed2-4cd8-b29f-e31853037a9e\") " Jul 21 17:11:42.514016 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513661 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:11:42.514016 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513940 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:11:42.514016 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.513674 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:11:42.514182 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.514042 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:11:42.514182 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.514049 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.514182 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.514100 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.514182 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.514116 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.514389 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.514346 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:11:42.515894 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.515865 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:11:42.516015 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.515892 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:11:42.516269 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.516242 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:11:42.516269 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.516252 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-kube-api-access-mb9dp" (OuterVolumeSpecName: "kube-api-access-mb9dp") pod "3a6c52cb-eed2-4cd8-b29f-e31853037a9e" (UID: "3a6c52cb-eed2-4cd8-b29f-e31853037a9e"). InnerVolumeSpecName "kube-api-access-mb9dp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:11:42.615195 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.615147 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.615195 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.615193 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mb9dp\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-kube-api-access-mb9dp\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.615195 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.615204 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.615195 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.615213 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.615481 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.615222 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:42.615481 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:42.615229 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a6c52cb-eed2-4cd8-b29f-e31853037a9e-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:11:43.350011 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:43.349976 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" event={"ID":"3a6c52cb-eed2-4cd8-b29f-e31853037a9e","Type":"ContainerDied","Data":"78718dba2ee43737389b7a45663fab367f02343e2b2bfe53c20758ac9a131f49"} Jul 21 17:11:43.350011 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:43.350003 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4" Jul 21 17:11:43.350475 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:43.350037 2557 scope.go:117] "RemoveContainer" containerID="9b5e224348869174e42bc8de1ed80600c4775ce95ddba8e729b2227899d08792" Jul 21 17:11:43.373001 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:43.372972 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4"] Jul 21 17:11:43.376723 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:43.376701 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-07bb43-openshift-default-f47c9ff9-qrcg4"] Jul 21 17:11:43.490661 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:43.490628 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" path="/var/lib/kubelet/pods/3a6c52cb-eed2-4cd8-b29f-e31853037a9e/volumes" Jul 21 17:11:59.208731 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:59.208695 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc"] Jul 21 17:11:59.209224 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:11:59.208945 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" containerID="cri-o://0e5d83d0b35344ef84cd0971a0a729c0b4b20b4e397ed3d489091c0d1bcad634" gracePeriod=30 Jul 21 17:12:04.460891 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.460860 2557 generic.go:356] "Generic (PLEG): container finished" podID="17651380-163e-4703-ae76-091c7f6b10f5" containerID="0e5d83d0b35344ef84cd0971a0a729c0b4b20b4e397ed3d489091c0d1bcad634" exitCode=0 Jul 21 17:12:04.461198 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.460935 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" event={"ID":"17651380-163e-4703-ae76-091c7f6b10f5","Type":"ContainerDied","Data":"0e5d83d0b35344ef84cd0971a0a729c0b4b20b4e397ed3d489091c0d1bcad634"} Jul 21 17:12:04.461198 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.461007 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" event={"ID":"17651380-163e-4703-ae76-091c7f6b10f5","Type":"ContainerDied","Data":"3fa5ee2fcdce9b097f5407efee3783257454337cd7f44f943e9b5b7d9692e2ca"} Jul 21 17:12:04.461198 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.461032 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fa5ee2fcdce9b097f5407efee3783257454337cd7f44f943e9b5b7d9692e2ca" Jul 21 17:12:04.461198 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.461050 2557 scope.go:117] "RemoveContainer" containerID="f91814674fa4e1ed9d5b33bed850e3f023736ebd2b9f96d7dd0ede5bc8870373" Jul 21 17:12:04.466303 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.466285 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:12:04.513449 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513416 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-socket\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.513664 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513467 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-istio-token\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.513664 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513490 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-envoy\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.513664 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513525 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-credential-socket\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.513664 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513577 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6fv5\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-kube-api-access-h6fv5\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.513664 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513611 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-data\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.513664 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513632 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17651380-163e-4703-ae76-091c7f6b10f5-istiod-ca-cert\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.514072 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513696 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17651380-163e-4703-ae76-091c7f6b10f5-istio-podinfo\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.514072 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513726 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-certs\") pod \"17651380-163e-4703-ae76-091c7f6b10f5\" (UID: \"17651380-163e-4703-ae76-091c7f6b10f5\") " Jul 21 17:12:04.514072 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513719 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:12:04.514072 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513847 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:12:04.514072 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.513955 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:12:04.514345 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.514131 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.514345 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.514151 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.514345 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.514165 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.514345 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.514196 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17651380-163e-4703-ae76-091c7f6b10f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:12:04.514645 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.514618 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:12:04.515780 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.515749 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:12:04.516215 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.516196 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-kube-api-access-h6fv5" (OuterVolumeSpecName: "kube-api-access-h6fv5") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "kube-api-access-h6fv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:12:04.516309 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.516295 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:12:04.516369 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.516321 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/17651380-163e-4703-ae76-091c7f6b10f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "17651380-163e-4703-ae76-091c7f6b10f5" (UID: "17651380-163e-4703-ae76-091c7f6b10f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:12:04.614885 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.614850 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.614885 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.614877 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.614885 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.614887 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6fv5\" (UniqueName: \"kubernetes.io/projected/17651380-163e-4703-ae76-091c7f6b10f5-kube-api-access-h6fv5\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.615114 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.614897 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/17651380-163e-4703-ae76-091c7f6b10f5-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.615114 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.614907 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/17651380-163e-4703-ae76-091c7f6b10f5-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:04.615114 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:04.614915 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/17651380-163e-4703-ae76-091c7f6b10f5-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 21 17:12:05.059059 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.059031 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4kkm6_492b1073-be50-405b-aef3-ae8c9407fc2f/manager/0.log" Jul 21 17:12:05.174726 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.174695 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qnd6g_b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5/manager/0.log" Jul 21 17:12:05.425456 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.425380 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jkxl5_2b4adb52-eeb3-4c58-98d8-94be5ae928e6/server/0.log" Jul 21 17:12:05.467133 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.467104 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc" Jul 21 17:12:05.507962 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.507929 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc"] Jul 21 17:12:05.512539 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.512510 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-1e5534-openshift-default-7bfbb48b59-gbphc"] Jul 21 17:12:05.898021 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.897993 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d658cff7b-frc49_4765e0fe-354c-44b6-9589-a56ca58799c4/copy-manifests/0.log" Jul 21 17:12:05.911026 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:05.911000 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d658cff7b-frc49_4765e0fe-354c-44b6-9589-a56ca58799c4/manager/0.log" Jul 21 17:12:06.386348 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:06.386312 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744205-6gcw9_df498052-6027-467b-a649-d77cece3769e/cleanup/0.log" Jul 21 17:12:06.497910 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:06.497878 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744220-j5k8r_5f7c3bb0-edf6-41cb-862a-e143e92b45f4/cleanup/0.log" Jul 21 17:12:07.489681 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:07.489649 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17651380-163e-4703-ae76-091c7f6b10f5" path="/var/lib/kubelet/pods/17651380-163e-4703-ae76-091c7f6b10f5/volumes" Jul 21 17:12:08.097544 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:08.097516 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lrb7d_bd983f9c-67c5-40d2-9176-7d0188e8446d/manager/0.log" Jul 21 17:12:08.670241 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:08.670196 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4w9ck_5f2e30d1-f72e-4771-8639-9d6970d378ff/manager/0.log" Jul 21 17:12:09.142379 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:09.142349 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf2fn_f813ed7c-954f-4567-9321-081bcee44a36/discovery/0.log" Jul 21 17:12:09.364441 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:09.364412 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcb5fbcb6-s62mq_1a700735-0ab6-4fca-b7a3-cce620939005/kube-auth-proxy/0.log" Jul 21 17:12:09.713851 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:09.713791 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ckvzg_e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef/payload-processing/0.log" Jul 21 17:12:09.820936 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:09.820907 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c99488556-jpl8w_cc2b1af9-464a-4817-8788-b5078e54c4c6/router/0.log" Jul 21 17:12:10.278052 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:10.278010 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-x2ftq_570120ea-b8c6-4e44-ab85-e77360155287/main/0.log" Jul 21 17:12:10.495091 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:10.495024 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68frz7h8_d4eac61d-9591-40a6-beb9-8480928d4c37/main/0.log" Jul 21 17:12:10.604732 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:10.604644 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ncd7n_9bb10eec-dea6-4761-91e9-a2960b023096/main/0.log" Jul 21 17:12:18.722046 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:18.722013 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9t8kx_44dab062-86b8-4849-9d19-75299742a580/global-pull-secret-syncer/0.log" Jul 21 17:12:18.891972 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:18.891929 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xdwg8_5791f270-f605-4255-8fad-d441db9d0be3/konnectivity-agent/0.log" Jul 21 17:12:18.913778 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:18.913749 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-151.ec2.internal_90ea7be93df65d66da652d9417af881e/haproxy/0.log" Jul 21 17:12:24.530193 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:24.530166 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-lrb7d_bd983f9c-67c5-40d2-9176-7d0188e8446d/manager/0.log" Jul 21 17:12:24.917023 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:24.916939 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-4w9ck_5f2e30d1-f72e-4771-8639-9d6970d378ff/manager/0.log" Jul 21 17:12:26.889074 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:26.889043 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-jxxfj_b369a7f2-8765-4858-9bb0-ab0f52838e21/cluster-monitoring-operator/0.log" Jul 21 17:12:26.916524 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:26.916493 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-jqqmt_a7575720-bd03-4d30-85a9-a05da5073932/kube-state-metrics/0.log" Jul 21 17:12:26.938405 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:26.938375 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-jqqmt_a7575720-bd03-4d30-85a9-a05da5073932/kube-rbac-proxy-main/0.log" Jul 21 17:12:26.964751 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:26.964718 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-jqqmt_a7575720-bd03-4d30-85a9-a05da5073932/kube-rbac-proxy-self/0.log" Jul 21 17:12:27.225458 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.225439 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h642n_74df7180-83e0-4993-9111-12fb70e0d1b4/node-exporter/0.log" Jul 21 17:12:27.247976 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.247952 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h642n_74df7180-83e0-4993-9111-12fb70e0d1b4/kube-rbac-proxy/0.log" Jul 21 17:12:27.274614 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.274591 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h642n_74df7180-83e0-4993-9111-12fb70e0d1b4/init-textfile/0.log" Jul 21 17:12:27.599886 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.599801 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-m8ng6_7a93b7bc-6795-47d6-a337-9c7504954d09/prometheus-operator/0.log" Jul 21 17:12:27.623437 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.623409 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-m8ng6_7a93b7bc-6795-47d6-a337-9c7504954d09/kube-rbac-proxy/0.log" Jul 21 17:12:27.648182 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.648156 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-hljbw_ee472008-8795-4ff5-adea-fa2adaeb187e/prometheus-operator-admission-webhook/0.log" Jul 21 17:12:27.780440 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.780407 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fbf7f598c-mnmnk_9b0156c2-594c-462c-ac56-50caf968fe96/thanos-query/0.log" Jul 21 17:12:27.802425 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.802396 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fbf7f598c-mnmnk_9b0156c2-594c-462c-ac56-50caf968fe96/kube-rbac-proxy-web/0.log" Jul 21 17:12:27.822465 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.822439 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fbf7f598c-mnmnk_9b0156c2-594c-462c-ac56-50caf968fe96/kube-rbac-proxy/0.log" Jul 21 17:12:27.865555 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.865485 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fbf7f598c-mnmnk_9b0156c2-594c-462c-ac56-50caf968fe96/prom-label-proxy/0.log" Jul 21 17:12:27.891809 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.891783 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fbf7f598c-mnmnk_9b0156c2-594c-462c-ac56-50caf968fe96/kube-rbac-proxy-rules/0.log" Jul 21 17:12:27.913832 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:27.913781 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7fbf7f598c-mnmnk_9b0156c2-594c-462c-ac56-50caf968fe96/kube-rbac-proxy-metrics/0.log" Jul 21 17:12:28.605933 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.605901 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt"] Jul 21 17:12:28.606317 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606305 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" Jul 21 17:12:28.606360 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606319 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" Jul 21 17:12:28.606360 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606330 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" Jul 21 17:12:28.606360 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606336 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" Jul 21 17:12:28.606360 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606342 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" Jul 21 17:12:28.606360 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606348 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" Jul 21 17:12:28.606360 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606356 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" Jul 21 17:12:28.606360 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606362 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" Jul 21 17:12:28.606573 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606420 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a6c52cb-eed2-4cd8-b29f-e31853037a9e" containerName="istio-proxy" Jul 21 17:12:28.606573 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.606427 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="17651380-163e-4703-ae76-091c7f6b10f5" containerName="istio-proxy" Jul 21 17:12:28.608556 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.608535 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.611799 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.611776 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qc9mq\"/\"openshift-service-ca.crt\"" Jul 21 17:12:28.611799 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.611792 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qc9mq\"/\"default-dockercfg-652bc\"" Jul 21 17:12:28.611799 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.611799 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qc9mq\"/\"kube-root-ca.crt\"" Jul 21 17:12:28.620914 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.620888 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt"] Jul 21 17:12:28.735945 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.735906 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n77cc\" (UniqueName: \"kubernetes.io/projected/36aa1ebd-cb22-486a-b555-c17fdc7413aa-kube-api-access-n77cc\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.736162 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.735953 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-sys\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.736162 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.736042 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-lib-modules\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.736162 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.736107 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-proc\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.736162 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.736145 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-podres\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837035 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-lib-modules\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837056 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-proc\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837085 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-podres\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n77cc\" (UniqueName: \"kubernetes.io/projected/36aa1ebd-cb22-486a-b555-c17fdc7413aa-kube-api-access-n77cc\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837164 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-sys\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837164 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-proc\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837193 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-lib-modules\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837237 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837236 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-sys\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.837502 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.837268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/36aa1ebd-cb22-486a-b555-c17fdc7413aa-podres\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.846951 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.846925 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n77cc\" (UniqueName: \"kubernetes.io/projected/36aa1ebd-cb22-486a-b555-c17fdc7413aa-kube-api-access-n77cc\") pod \"perf-node-gather-daemonset-89xbt\" (UID: \"36aa1ebd-cb22-486a-b555-c17fdc7413aa\") " pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:28.920205 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:28.920116 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:29.073035 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.073005 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt"] Jul 21 17:12:29.075684 ip-10-0-132-151 kubenswrapper[2557]: W0721 17:12:29.075655 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod36aa1ebd_cb22_486a_b555_c17fdc7413aa.slice/crio-b9d26cbe014baf2dd0a7aa4261e690e207f8dc65988c6f22fe07f7e81ce28eb8 WatchSource:0}: Error finding container b9d26cbe014baf2dd0a7aa4261e690e207f8dc65988c6f22fe07f7e81ce28eb8: Status 404 returned error can't find the container with id b9d26cbe014baf2dd0a7aa4261e690e207f8dc65988c6f22fe07f7e81ce28eb8 Jul 21 17:12:29.117184 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.117153 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-4q94l_06057afc-c6f1-4735-a59b-20e7ada2f7c9/networking-console-plugin/0.log" Jul 21 17:12:29.596205 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.596162 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" event={"ID":"36aa1ebd-cb22-486a-b555-c17fdc7413aa","Type":"ContainerStarted","Data":"be2ee139cb0b9ee80964612f2218328f18e342bf284ee52f460f3036248d1ea2"} Jul 21 17:12:29.596417 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.596219 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" event={"ID":"36aa1ebd-cb22-486a-b555-c17fdc7413aa","Type":"ContainerStarted","Data":"b9d26cbe014baf2dd0a7aa4261e690e207f8dc65988c6f22fe07f7e81ce28eb8"} Jul 21 17:12:29.596417 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.596249 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:29.615971 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.615920 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" podStartSLOduration=1.61590491 podStartE2EDuration="1.61590491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:12:29.614735313 +0000 UTC m=+2862.653570977" watchObservedRunningTime="2026-07-21 17:12:29.61590491 +0000 UTC m=+2862.654740572" Jul 21 17:12:29.709683 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.709652 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/1.log" Jul 21 17:12:29.715984 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:29.715957 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-59rmv_bc389b39-7c81-4b6a-aba7-920c43ad61b4/console-operator/2.log" Jul 21 17:12:30.228019 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:30.227992 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-mgqrm_59fdf7f1-4d2e-4cad-93c0-01eb08ae20b9/download-server/0.log" Jul 21 17:12:30.826103 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:30.826071 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-ml2gj_43b75aa3-b7f9-4aed-84bc-98e5f73d659a/volume-data-source-validator/0.log" Jul 21 17:12:31.909909 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:31.909878 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cxndm_6669ba74-1154-4860-b7d8-8f3482d4255b/dns/0.log" Jul 21 17:12:31.931875 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:31.931816 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cxndm_6669ba74-1154-4860-b7d8-8f3482d4255b/kube-rbac-proxy/0.log" Jul 21 17:12:32.034046 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:32.034018 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dwl4v_9d59dd5a-8f0a-4ad5-b3e2-d1e89eff7816/dns-node-resolver/0.log" Jul 21 17:12:32.658358 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:32.658328 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-js8z5_96c383c4-32ae-4252-bda3-6575b8d24b76/node-ca/0.log" Jul 21 17:12:33.722275 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:33.722239 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf2fn_f813ed7c-954f-4567-9321-081bcee44a36/discovery/0.log" Jul 21 17:12:33.770626 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:33.770596 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcb5fbcb6-s62mq_1a700735-0ab6-4fca-b7a3-cce620939005/kube-auth-proxy/0.log" Jul 21 17:12:33.886749 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:33.886717 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ckvzg_e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef/payload-processing/0.log" Jul 21 17:12:33.907243 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:33.907220 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c99488556-jpl8w_cc2b1af9-464a-4817-8788-b5078e54c4c6/router/0.log" Jul 21 17:12:34.579429 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:34.579398 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7vszk_cf308e21-a887-4859-93c3-5dfb60a312ef/serve-healthcheck-canary/0.log" Jul 21 17:12:35.093289 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:35.093260 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-h2cdv_23d93917-4b64-4728-a183-f39e4530367a/insights-operator/0.log" Jul 21 17:12:35.094236 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:35.094215 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-h2cdv_23d93917-4b64-4728-a183-f39e4530367a/insights-operator/1.log" Jul 21 17:12:35.118484 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:35.118456 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2frkk_addf81be-facb-4540-8b4c-4960f292deb8/kube-rbac-proxy/0.log" Jul 21 17:12:35.139904 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:35.139884 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2frkk_addf81be-facb-4540-8b4c-4960f292deb8/exporter/0.log" Jul 21 17:12:35.162062 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:35.162037 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2frkk_addf81be-facb-4540-8b4c-4960f292deb8/extractor/0.log" Jul 21 17:12:35.610493 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:35.610460 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qc9mq/perf-node-gather-daemonset-89xbt" Jul 21 17:12:37.823689 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:37.823658 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4kkm6_492b1073-be50-405b-aef3-ae8c9407fc2f/manager/0.log" Jul 21 17:12:38.014361 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:38.014329 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-qnd6g_b4397e1d-6b9a-4f54-acce-9fa8b0dfdff5/manager/0.log" Jul 21 17:12:38.158479 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:38.158454 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-jkxl5_2b4adb52-eeb3-4c58-98d8-94be5ae928e6/server/0.log" Jul 21 17:12:38.427656 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:38.427579 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d658cff7b-frc49_4765e0fe-354c-44b6-9589-a56ca58799c4/manager/0.log" Jul 21 17:12:38.448652 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:38.448624 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7d658cff7b-frc49_4765e0fe-354c-44b6-9589-a56ca58799c4/copy-manifests/0.log" Jul 21 17:12:39.185914 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:39.185880 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf2fn_f813ed7c-954f-4567-9321-081bcee44a36/discovery/0.log" Jul 21 17:12:39.226391 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:39.226368 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcb5fbcb6-s62mq_1a700735-0ab6-4fca-b7a3-cce620939005/kube-auth-proxy/0.log" Jul 21 17:12:39.317451 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:39.317424 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ckvzg_e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef/payload-processing/0.log" Jul 21 17:12:39.333132 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:39.333107 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c99488556-jpl8w_cc2b1af9-464a-4817-8788-b5078e54c4c6/router/0.log" Jul 21 17:12:40.717657 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:40.717578 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-cf2fn_f813ed7c-954f-4567-9321-081bcee44a36/discovery/0.log" Jul 21 17:12:40.750332 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:40.750309 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6dcb5fbcb6-s62mq_1a700735-0ab6-4fca-b7a3-cce620939005/kube-auth-proxy/0.log" Jul 21 17:12:40.835501 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:40.835468 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-ckvzg_e9db0f50-ae6b-4eab-bc2a-1238b9b5b1ef/payload-processing/0.log" Jul 21 17:12:40.849997 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:40.849972 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-c99488556-jpl8w_cc2b1af9-464a-4817-8788-b5078e54c4c6/router/0.log" Jul 21 17:12:45.225160 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:45.225121 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-qknkl_e5edd2df-971a-4809-b26d-51110cac4ead/kube-storage-version-migrator-operator/1.log" Jul 21 17:12:45.225954 ip-10-0-132-151 kubenswrapper[2557]: I0721 17:12:45.225936 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-qknkl_e5edd2df-971a-4809-b26d-51110cac4ead/kube-storage-version-migrator-operator/0.log"