Jul 20 19:25:24.281675 ip-10-0-128-93 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 20 19:25:24.281685 ip-10-0-128-93 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 20 19:25:24.281692 ip-10-0-128-93 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 20 19:25:24.281907 ip-10-0-128-93 systemd[1]: Failed to start Kubernetes Kubelet. Jul 20 19:25:34.466951 ip-10-0-128-93 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 20 19:25:34.466967 ip-10-0-128-93 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot b341b02e81ab4c438a6f0195a3ac76b6 -- Jul 20 19:27:22.249712 ip-10-0-128-93 systemd[1]: Starting Kubernetes Kubelet... Jul 20 19:27:22.695967 ip-10-0-128-93 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 19:27:22.695967 ip-10-0-128-93 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 19:27:22.695967 ip-10-0-128-93 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 19:27:22.695967 ip-10-0-128-93 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 19:27:22.695967 ip-10-0-128-93 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 19:27:22.696805 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.696718 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 19:27:22.704161 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704144 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:22.704161 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704162 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704166 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704169 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704173 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704176 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704179 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704182 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704184 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704187 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704190 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704195 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704199 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704202 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704205 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704209 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704212 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704214 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704217 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704219 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:22.704231 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704222 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704225 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704227 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704229 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704232 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704235 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704238 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704241 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704244 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704246 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704250 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704253 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704256 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704259 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704262 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704264 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704267 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704270 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704273 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:22.704688 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704275 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704278 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704281 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704284 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704286 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704289 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704291 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704294 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704298 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704302 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704306 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704309 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704312 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704314 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704317 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704319 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704322 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704325 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704328 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:22.705399 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704331 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704333 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704337 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704340 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704342 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704345 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704347 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704350 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704353 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704356 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704359 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704361 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704364 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704367 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704369 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704372 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704375 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704377 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704380 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704382 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:22.706146 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704385 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704388 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704391 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704393 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704396 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704398 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704401 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704404 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704407 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704409 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704412 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704427 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704430 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704432 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704435 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704438 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704440 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704443 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704446 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704448 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:22.706864 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.704451 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706303 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706323 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706330 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706335 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706347 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706352 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706357 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706361 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706366 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706370 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706375 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706379 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706384 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706389 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706394 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706398 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706403 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706433 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706440 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:22.707335 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706444 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706449 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706453 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706458 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706462 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706467 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706471 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706475 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706483 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706489 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706499 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706504 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706509 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706514 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706520 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706525 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706530 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706534 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706538 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:22.707866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706543 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706547 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706552 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706556 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706565 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706570 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706574 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706578 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706592 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706598 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706603 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706608 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706622 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706627 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706665 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706811 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706818 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706821 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706824 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:22.708323 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706827 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706832 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706835 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706838 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706841 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706844 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706846 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706849 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706856 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706859 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706861 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706864 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706867 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706870 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706874 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706877 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706880 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706883 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706886 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706888 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:22.708806 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706891 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706893 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706896 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706898 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706901 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706903 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706906 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706908 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706911 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706913 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706916 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706918 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706921 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706923 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706926 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706928 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706931 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706935 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706941 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:22.709285 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706943 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706946 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.706951 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707360 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707366 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707370 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707373 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707376 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707379 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707382 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707385 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707388 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707390 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707393 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707396 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707398 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707402 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707405 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707408 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:22.709761 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707410 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707413 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707430 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707433 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707435 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707438 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707441 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707444 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707447 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707449 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707452 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707454 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707457 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707459 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707462 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707464 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707467 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707470 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707472 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:22.710208 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707475 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707478 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707481 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707484 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707486 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707489 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707491 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707493 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707496 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707498 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707501 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707504 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707506 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707509 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707511 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707515 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707519 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707522 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707526 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:22.710684 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707528 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707531 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707534 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707537 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707540 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707543 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707545 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707548 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707551 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707553 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707556 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707558 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707561 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707564 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707566 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707569 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707571 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707574 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707576 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707579 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:22.711133 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707582 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707585 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707587 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707590 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707592 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707595 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707597 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707599 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707602 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707605 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707607 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707610 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707613 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707616 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707618 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707621 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707623 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707626 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707628 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707630 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:22.711619 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707633 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707636 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707640 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707643 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.707646 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707711 2573 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707718 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707724 2573 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707729 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707733 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707736 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707740 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707745 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707748 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707751 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707755 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707759 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707762 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707765 2573 flags.go:64] FLAG: --cgroup-root="" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707768 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707770 2573 flags.go:64] FLAG: --client-ca-file="" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707774 2573 flags.go:64] FLAG: --cloud-provider="external" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707777 2573 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 19:27:22.712082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707784 2573 flags.go:64] FLAG: --cluster-domain="" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707787 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707790 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707794 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707797 2573 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707806 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707809 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707812 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707816 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707819 2573 flags.go:64] FLAG: --contention-profiling="false" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707822 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707825 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707828 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707831 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707835 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707838 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707841 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707844 2573 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707847 2573 flags.go:64] FLAG: --enable-server="true" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707849 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707853 2573 flags.go:64] FLAG: --event-burst="100" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707856 2573 flags.go:64] FLAG: --event-qps="50" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707859 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707862 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 19:27:22.712690 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707865 2573 flags.go:64] FLAG: --eviction-hard="" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707869 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707872 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707875 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707878 2573 flags.go:64] FLAG: --eviction-soft="" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707881 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707885 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707888 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707894 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707897 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707899 2573 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707902 2573 flags.go:64] FLAG: --feature-gates="" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707906 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707909 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707912 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707915 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707918 2573 flags.go:64] FLAG: --healthz-port="10248" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707922 2573 flags.go:64] FLAG: --help="false" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707925 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-128-93.ec2.internal" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707928 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707931 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707933 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707937 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 19:27:22.713242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707940 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707943 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707946 2573 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707949 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707951 2573 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707954 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707957 2573 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707961 2573 flags.go:64] FLAG: --kube-reserved="" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707964 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707966 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707969 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707972 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707975 2573 flags.go:64] FLAG: --lock-file="" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707978 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707981 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707984 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707990 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707994 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.707997 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708000 2573 flags.go:64] FLAG: --logging-format="text" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708003 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708006 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708009 2573 flags.go:64] FLAG: --manifest-url="" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708012 2573 flags.go:64] FLAG: --manifest-url-header="" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708016 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 19:27:22.713848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708019 2573 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708023 2573 flags.go:64] FLAG: --max-pods="110" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708026 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708029 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708032 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708035 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708038 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708041 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708043 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708050 2573 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708053 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708056 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708059 2573 flags.go:64] FLAG: --pod-cidr="" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708062 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708068 2573 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708070 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708074 2573 flags.go:64] FLAG: --pods-per-core="0" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708077 2573 flags.go:64] FLAG: --port="10250" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708080 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708083 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02b1732a411959fc7" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708086 2573 flags.go:64] FLAG: --qos-reserved="" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708089 2573 flags.go:64] FLAG: --read-only-port="10255" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708092 2573 flags.go:64] FLAG: --register-node="true" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708095 2573 flags.go:64] FLAG: --register-with-taints="" Jul 20 19:27:22.714513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708099 2573 flags.go:64] FLAG: --registry-burst="10" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708103 2573 flags.go:64] FLAG: --registry-qps="5" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708107 2573 flags.go:64] FLAG: --reserved-cpus="" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708109 2573 flags.go:64] FLAG: --reserved-memory="" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708113 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708116 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708119 2573 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708122 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708125 2573 flags.go:64] FLAG: --runonce="false" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708135 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708140 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708143 2573 flags.go:64] FLAG: --seccomp-default="false" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708151 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708154 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708157 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708161 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708164 2573 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708167 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708170 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708173 2573 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708175 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708178 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708181 2573 flags.go:64] FLAG: --system-cgroups="" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708184 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708189 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 19:27:22.715092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708192 2573 flags.go:64] FLAG: --tls-cert-file="" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708195 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708200 2573 flags.go:64] FLAG: --tls-min-version="" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708202 2573 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708205 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708208 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708211 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708213 2573 flags.go:64] FLAG: --v="2" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708219 2573 flags.go:64] FLAG: --version="false" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708224 2573 flags.go:64] FLAG: --vmodule="" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708228 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.708231 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708343 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708347 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708350 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708353 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708356 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708359 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708361 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708365 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708368 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708370 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:22.715723 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708373 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708376 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708378 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708381 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708383 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708386 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708389 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708391 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708394 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708396 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708399 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708401 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708404 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708407 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708409 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708411 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708428 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708431 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708435 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:22.716286 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708438 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708441 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708444 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708446 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708449 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708452 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708456 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708459 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708462 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708465 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708469 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708472 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708475 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708477 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708480 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708483 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708486 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708489 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708492 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708494 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:22.716757 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708497 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708499 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708502 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708504 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708507 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708509 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708512 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708514 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708517 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708519 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708522 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708526 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708528 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708531 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708534 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708537 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708541 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708544 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708547 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708550 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:22.717255 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708552 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708555 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708559 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708561 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708564 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708566 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708569 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708572 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708575 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708577 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708580 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708582 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708585 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708587 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708590 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708592 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708596 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708598 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708601 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708603 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:22.717836 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708606 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708608 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708611 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708614 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708617 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708620 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708623 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708628 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708631 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.708634 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.709319 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.715911 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.715926 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.715944 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 19:27:22.718316 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.715950 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716004 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716009 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716014 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716017 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716020 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716023 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716026 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716029 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716032 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716035 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716038 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716041 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716045 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716049 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716052 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716055 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716058 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:22.718693 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716062 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716066 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716069 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716072 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716074 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716077 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716079 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716083 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716085 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716088 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716091 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716094 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716097 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716100 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716102 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716105 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716107 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716111 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716114 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:22.719119 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716117 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716119 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716122 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716125 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716127 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716139 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716142 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716145 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716147 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716150 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716152 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716155 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716158 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716161 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716163 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716166 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716169 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716171 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716174 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716177 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:22.719599 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716179 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716183 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716186 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716188 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716191 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716195 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716198 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716200 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716203 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716205 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716208 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716211 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716213 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716216 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716218 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716221 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716223 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716226 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716229 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716231 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:22.720063 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716233 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716236 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716238 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716241 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716244 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716246 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716249 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716251 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716254 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716256 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716258 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716261 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716263 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716266 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716269 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716272 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716275 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716277 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716280 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716283 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:22.720632 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716285 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716288 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716291 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.716296 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716404 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716408 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716411 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716429 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716433 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716436 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716438 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716441 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716444 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716447 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716450 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 19:27:22.721129 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716452 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716454 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716457 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716460 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716462 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716464 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716467 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716470 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716473 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716475 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716478 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716481 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716484 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716487 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716489 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716492 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716494 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716497 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716499 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716502 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 19:27:22.721518 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716505 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716509 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716512 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716514 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716517 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716520 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716522 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716525 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716528 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716532 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716535 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716537 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716540 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716543 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716545 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716548 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716550 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716553 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716555 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 19:27:22.722098 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716557 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716560 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716563 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716565 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716568 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716570 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716573 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716576 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716579 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716581 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716585 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716588 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716591 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716594 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716597 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716599 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716602 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716604 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716607 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 19:27:22.722579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716609 2573 feature_gate.go:328] unrecognized feature gate: Example Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716612 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716615 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716617 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716620 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716622 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716625 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716627 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716630 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716633 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716635 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716638 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716640 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716643 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716645 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716648 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716650 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716653 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716656 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716658 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 19:27:22.723337 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716662 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716664 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716667 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716670 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716672 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716674 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716677 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716679 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716682 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:22.716684 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.716689 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.716793 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.720202 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.721687 2573 server.go:1019] "Starting client certificate rotation" Jul 20 19:27:22.723948 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.721799 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 19:27:22.724450 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.721849 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 19:27:22.724591 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.724576 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 19:27:22.728540 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.728523 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 19:27:22.739805 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.739785 2573 log.go:25] "Validated CRI v1 runtime API" Jul 20 19:27:22.743480 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.743462 2573 log.go:25] "Validated CRI v1 image API" Jul 20 19:27:22.744269 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.744247 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 19:27:22.748474 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.748454 2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9697f851-01ac-403a-85e1-697b5801c01a:/dev/nvme0n1p4 cb21363f-58f5-4f32-860c-006ad69d8270:/dev/nvme0n1p3] Jul 20 19:27:22.748530 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.748474 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 19:27:22.753823 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.753718 2573 manager.go:216] Machine: {Timestamp:2026-07-20 19:27:22.752403553 +0000 UTC m=+0.385892070 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098513 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec247a148fbf0268143ff998d9ece49f SystemUUID:ec247a14-8fbf-0268-143f-f998d9ece49f BootID:b341b02e-81ab-4c43-8a6f-0195a3ac76b6 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:57:d7:82:55:97 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:57:d7:82:55:97 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:95:f1:40:1a:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 19:27:22.753823 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.753820 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 19:27:22.753950 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.753895 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 19:27:22.754471 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.754454 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 19:27:22.756955 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.756924 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 19:27:22.757101 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.756958 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-93.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 19:27:22.757868 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.757847 2573 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 19:27:22.757868 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.757871 2573 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 19:27:22.757939 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.757879 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 19:27:22.758617 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.758607 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 19:27:22.759739 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.759730 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 19:27:22.762718 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.762707 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 20 19:27:22.762844 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.762834 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 19:27:22.762876 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.762867 2573 kubelet.go:486] "Attempting to sync node with API server" Jul 20 19:27:22.762908 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.762878 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 19:27:22.762908 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.762892 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 19:27:22.762908 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.762901 2573 kubelet.go:398] "Adding apiserver pod source" Jul 20 19:27:22.763008 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.762910 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 19:27:22.763274 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.763260 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 19:27:22.763336 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.763294 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 19:27:22.765788 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.765774 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 19:27:22.767680 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.767666 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 19:27:22.767726 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.767697 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 19:27:22.769226 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769215 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 19:27:22.769265 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769230 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 19:27:22.769265 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769237 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 19:27:22.769265 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769243 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 19:27:22.769265 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769251 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 19:27:22.769265 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769257 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 19:27:22.769265 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769265 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 19:27:22.769437 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769270 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 19:27:22.769437 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769277 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 19:27:22.769437 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769282 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 19:27:22.769437 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769295 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 19:27:22.769437 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.769302 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 19:27:22.771083 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.771058 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 19:27:22.771083 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.771078 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 19:27:22.774267 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.774250 2573 server.go:1268] "Started kubelet" Jul 20 19:27:22.774479 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.774434 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 19:27:22.774602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.774499 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 19:27:22.774764 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.774729 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 19:27:22.775086 ip-10-0-128-93 systemd[1]: Started Kubernetes Kubelet. Jul 20 19:27:22.775899 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.775877 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 19:27:22.775978 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:22.775948 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 19:27:22.776094 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:22.776049 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 19:27:22.776357 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.776123 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 19:27:22.776862 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:22.776848 2573 server.go:310] "Adding debug handlers to kubelet server" Jul 20 19:27:24.429436 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.429405 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 19:27:24.432848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.432831 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 19:27:24.432932 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.432832 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 19:27:24.432979 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.432933 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:24.433183 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.432218 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-93.ec2.internal.18c416178e5cdd99 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-93.ec2.internal,UID:ip-10-0-128-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-93.ec2.internal,},FirstTimestamp:2026-07-20 19:27:22.774068633 +0000 UTC m=+0.407557128,LastTimestamp:2026-07-20 19:27:22.774068633 +0000 UTC m=+0.407557128,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-93.ec2.internal,}" Jul 20 19:27:24.433433 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.433409 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 19:27:24.433433 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.433433 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 19:27:24.433554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.433484 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 19:27:24.433603 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.433580 2573 reconstruct.go:97] "Volume reconstruction finished" Jul 20 19:27:24.433603 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.433592 2573 reconciler.go:29] "Reconciler: start to sync state" Jul 20 19:27:24.433836 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.433811 2573 factory.go:55] Registering systemd factory Jul 20 19:27:24.433836 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.433829 2573 factory.go:223] Registration of the systemd container factory successfully Jul 20 19:27:24.434090 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.434075 2573 factory.go:153] Registering CRI-O factory Jul 20 19:27:24.434090 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.434090 2573 factory.go:223] Registration of the crio container factory successfully Jul 20 19:27:24.434189 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.434176 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 19:27:24.434239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.434198 2573 factory.go:103] Registering Raw factory Jul 20 19:27:24.434239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.434210 2573 manager.go:1196] Started watching for new ooms in manager Jul 20 19:27:24.434549 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.434538 2573 manager.go:349] Starting recovery of all containers Jul 20 19:27:24.442384 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.442352 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 19:27:24.442563 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.442512 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 19:27:24.444653 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.444635 2573 manager.go:354] Recovery completed Jul 20 19:27:24.449227 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.449210 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:24.451109 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.451088 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:24.451184 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.451127 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:24.451184 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.451139 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:24.451652 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.451637 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 19:27:24.451652 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.451651 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 19:27:24.451769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.451667 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:24.451769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.451670 2573 state_mem.go:36] "Initialized new in-memory state store" Jul 20 19:27:24.452363 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.452349 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:24.452428 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.452374 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:24.452428 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.452383 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:24.452965 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.452863 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-93.ec2.internal.18c41617f252791a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-93.ec2.internal,UID:ip-10-0-128-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-93.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-93.ec2.internal,},FirstTimestamp:2026-07-20 19:27:24.451109146 +0000 UTC m=+0.434402095,LastTimestamp:2026-07-20 19:27:24.451109146 +0000 UTC m=+0.434402095,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-93.ec2.internal,}" Jul 20 19:27:24.453074 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.453064 2573 policy_none.go:49] "None policy: Start" Jul 20 19:27:24.453116 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.453080 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 19:27:24.453116 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.453090 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 19:27:24.454133 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.454119 2573 policy_none.go:47] "Start" Jul 20 19:27:24.459236 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.459021 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-49hpv" Jul 20 19:27:24.463890 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.463824 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-93.ec2.internal.18c41617f252d7f7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-93.ec2.internal,UID:ip-10-0-128-93.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-93.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-93.ec2.internal,},FirstTimestamp:2026-07-20 19:27:24.451133431 +0000 UTC m=+0.434426377,LastTimestamp:2026-07-20 19:27:24.451133431 +0000 UTC m=+0.434426377,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-93.ec2.internal,}" Jul 20 19:27:24.466886 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.466871 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-49hpv" Jul 20 19:27:24.479327 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.479176 2573 manager.go:337] "Starting Device Plugin manager" Jul 20 19:27:24.479415 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.479346 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 19:27:24.479415 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.479358 2573 server.go:87] "Starting device plugin registration server" Jul 20 19:27:24.479596 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.479585 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 19:27:24.479644 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.479599 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 19:27:24.479737 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.479720 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 19:27:24.479809 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.479797 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 19:27:24.479893 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.479810 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 19:27:24.480321 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.480304 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 19:27:24.480390 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.480345 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:24.528968 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.528936 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 19:27:24.530156 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.530138 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 19:27:24.530228 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.530187 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 19:27:24.530228 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.530214 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 19:27:24.530332 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.530256 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 19:27:24.533332 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.533312 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:24.580512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.580481 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:24.581462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.581445 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:24.581543 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.581477 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:24.581543 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.581488 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:24.581543 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.581511 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.590053 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.590032 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.590102 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.590057 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-93.ec2.internal\": node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:24.613540 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.613520 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:24.630598 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.630566 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal"] Jul 20 19:27:24.630675 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.630648 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:24.632156 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.632128 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:24.632222 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.632168 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:24.632222 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.632178 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:24.633340 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.633329 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:24.633448 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.633434 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.633489 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.633473 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:24.633886 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.633863 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/694a9c0ad5d708ffa615de056b509f7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal\" (UID: \"694a9c0ad5d708ffa615de056b509f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.633961 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.633892 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/694a9c0ad5d708ffa615de056b509f7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal\" (UID: \"694a9c0ad5d708ffa615de056b509f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.633961 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.633916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5c31779b826e1e93da8d4838893e9654-config\") pod \"kube-apiserver-proxy-ip-10-0-128-93.ec2.internal\" (UID: \"5c31779b826e1e93da8d4838893e9654\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.634057 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.634039 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:24.634101 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.634070 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:24.634101 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.634079 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:24.634161 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.634046 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:24.634161 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.634138 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:24.634161 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.634147 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:24.635224 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.635211 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.635268 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.635235 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 19:27:24.635972 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.635960 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientMemory" Jul 20 19:27:24.636021 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.635982 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 19:27:24.636021 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.635991 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeHasSufficientPID" Jul 20 19:27:24.661249 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.661232 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-93.ec2.internal\" not found" node="ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.665815 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.665796 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-93.ec2.internal\" not found" node="ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.714111 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.714085 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:24.734054 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.734024 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/694a9c0ad5d708ffa615de056b509f7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal\" (UID: \"694a9c0ad5d708ffa615de056b509f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.734166 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.734062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/694a9c0ad5d708ffa615de056b509f7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal\" (UID: \"694a9c0ad5d708ffa615de056b509f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.734166 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.734066 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/694a9c0ad5d708ffa615de056b509f7a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal\" (UID: \"694a9c0ad5d708ffa615de056b509f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.734166 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.734087 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5c31779b826e1e93da8d4838893e9654-config\") pod \"kube-apiserver-proxy-ip-10-0-128-93.ec2.internal\" (UID: \"5c31779b826e1e93da8d4838893e9654\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.734166 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.734123 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5c31779b826e1e93da8d4838893e9654-config\") pod \"kube-apiserver-proxy-ip-10-0-128-93.ec2.internal\" (UID: \"5c31779b826e1e93da8d4838893e9654\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.734296 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.734168 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/694a9c0ad5d708ffa615de056b509f7a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal\" (UID: \"694a9c0ad5d708ffa615de056b509f7a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.814395 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.814371 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:24.915052 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:24.914973 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:24.964442 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.964412 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:24.968485 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:24.968217 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" Jul 20 19:27:25.015934 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:25.015906 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:25.116398 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:25.116372 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:25.216869 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:25.216814 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:25.317395 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:25.317367 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:25.370757 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.370732 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 19:27:25.371164 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.370875 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 19:27:25.417867 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:25.417840 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-93.ec2.internal\" not found" Jul 20 19:27:25.422792 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.422763 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:25.433753 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.433731 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" Jul 20 19:27:25.433894 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.433741 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 19:27:25.443558 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.443541 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 19:27:25.448274 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.448257 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 19:27:25.449797 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.449780 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" Jul 20 19:27:25.459063 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.459044 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 19:27:25.471161 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.471106 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 19:22:24 +0000 UTC" deadline="2027-12-13 21:40:29.135307876 +0000 UTC" Jul 20 19:27:25.471161 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.471134 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12266h13m3.664177264s" Jul 20 19:27:25.474165 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.474146 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hd75m" Jul 20 19:27:25.483014 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.482998 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hd75m" Jul 20 19:27:25.528994 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:25.528967 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod694a9c0ad5d708ffa615de056b509f7a.slice/crio-b9e6756c1318589eca1fea14872c48e08278b09fd39634318aa951ae165cda4c WatchSource:0}: Error finding container b9e6756c1318589eca1fea14872c48e08278b09fd39634318aa951ae165cda4c: Status 404 returned error can't find the container with id b9e6756c1318589eca1fea14872c48e08278b09fd39634318aa951ae165cda4c Jul 20 19:27:25.532037 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:25.531962 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c31779b826e1e93da8d4838893e9654.slice/crio-c78b7580f881ed6a69b2cc1a4423bee4923b8b4a4f28063495a8910ddb0f8e3a WatchSource:0}: Error finding container c78b7580f881ed6a69b2cc1a4423bee4923b8b4a4f28063495a8910ddb0f8e3a: Status 404 returned error can't find the container with id c78b7580f881ed6a69b2cc1a4423bee4923b8b4a4f28063495a8910ddb0f8e3a Jul 20 19:27:25.533427 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.533331 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" event={"ID":"694a9c0ad5d708ffa615de056b509f7a","Type":"ContainerStarted","Data":"b9e6756c1318589eca1fea14872c48e08278b09fd39634318aa951ae165cda4c"} Jul 20 19:27:25.533785 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.533768 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 19:27:25.568785 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.568761 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:25.979579 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:25.979554 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:26.414631 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.414591 2573 apiserver.go:52] "Watching apiserver" Jul 20 19:27:26.420181 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.420158 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 19:27:26.420602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.420576 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-qxwxz","openshift-image-registry/node-ca-ng274","openshift-multus/multus-additional-cni-plugins-wd7w8","openshift-network-diagnostics/network-check-target-rcc5g","openshift-network-operator/iptables-alerter-h8qk9","kube-system/global-pull-secret-syncer-pw8fv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q","openshift-cluster-node-tuning-operator/tuned-tmqdj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal","openshift-multus/multus-gnnjj","openshift-multus/network-metrics-daemon-qndtv","openshift-ovn-kubernetes/ovnkube-node-2vrt6","kube-system/konnectivity-agent-vxjjl","kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal"] Jul 20 19:27:26.422172 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.422135 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.423890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.423867 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.424369 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.424350 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 19:27:26.424513 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.424496 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 19:27:26.424641 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.424629 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 19:27:26.425015 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.424997 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j4c8d\"" Jul 20 19:27:26.425336 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.425319 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.425967 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.425786 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-94spn\"" Jul 20 19:27:26.426582 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.426565 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 19:27:26.427996 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.427810 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:26.427996 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.427888 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:26.428395 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.428322 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.428395 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.428334 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 19:27:26.428519 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.428391 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 19:27:26.429039 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.428798 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 19:27:26.429039 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.428937 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-brbnm\"" Jul 20 19:27:26.429039 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.428995 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 19:27:26.429224 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.429196 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 19:27:26.429279 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.429255 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 19:27:26.429817 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.429690 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.429817 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.429766 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:26.431075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.431056 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.431793 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.431775 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 19:27:26.431900 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.431879 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qt5x6\"" Jul 20 19:27:26.431985 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.431970 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:27:26.432670 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.432654 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.433121 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.433102 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 19:27:26.433748 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.433707 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:27:26.434099 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.434064 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 19:27:26.434180 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.434152 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kpbm7\"" Jul 20 19:27:26.434180 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.434165 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.434522 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.434498 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 19:27:26.434752 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.434681 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sqmkm\"" Jul 20 19:27:26.434854 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.434776 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 19:27:26.436098 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.436051 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.437052 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.437034 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:26.437140 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.437096 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:26.437495 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.437471 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 19:27:26.437570 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.437528 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4h977\"" Jul 20 19:27:26.437742 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.437724 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 19:27:26.438078 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.437843 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 19:27:26.439848 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.439826 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-c7nqd\"" Jul 20 19:27:26.440115 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.440098 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 19:27:26.444031 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d00559d-ac72-41c0-ab2d-68879013c63a-hosts-file\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.444124 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444058 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-system-cni-dir\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.444177 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444147 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.444383 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444366 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-kubernetes\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.444446 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444400 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-lib-modules\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.444499 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444462 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-host\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.444550 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65bsd\" (UniqueName: \"kubernetes.io/projected/0d00559d-ac72-41c0-ab2d-68879013c63a-kube-api-access-65bsd\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.444601 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444549 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:26.444672 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444644 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c51a827b-a50d-479f-a6a2-e467ed632aae-dbus\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.444732 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444685 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-cni-binary-copy\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.444732 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444718 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d00559d-ac72-41c0-ab2d-68879013c63a-tmp-dir\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.444829 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444743 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnrr4\" (UniqueName: \"kubernetes.io/projected/a4a5474d-673d-433f-97b4-ad599865fa24-kube-api-access-mnrr4\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.444829 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444773 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-host\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.444829 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444820 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.444970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444853 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.444970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frdlf\" (UniqueName: \"kubernetes.io/projected/e25cc402-8092-42b7-89c9-54619d601351-kube-api-access-frdlf\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.444970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444886 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.444970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444948 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-os-release\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.445158 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.444986 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7c63268-8eba-4619-9675-7ea82bcba390-tmp\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.445158 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445025 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e7344064-0d35-4e15-abae-379facb8bff0-agent-certs\") pod \"konnectivity-agent-vxjjl\" (UID: \"e7344064-0d35-4e15-abae-379facb8bff0\") " pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.445158 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445063 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9nwn\" (UniqueName: \"kubernetes.io/projected/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-kube-api-access-w9nwn\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.445158 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445113 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.445158 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445138 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.445387 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.445387 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445225 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-modprobe-d\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.445387 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445254 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e7344064-0d35-4e15-abae-379facb8bff0-konnectivity-ca\") pod \"konnectivity-agent-vxjjl\" (UID: \"e7344064-0d35-4e15-abae-379facb8bff0\") " pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.445387 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4a5474d-673d-433f-97b4-ad599865fa24-iptables-alerter-script\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.445578 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445464 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4a5474d-673d-433f-97b4-ad599865fa24-host-slash\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.445578 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445529 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-serviceca\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.445704 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.445704 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445653 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-cnibin\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.445799 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klsfx\" (UniqueName: \"kubernetes.io/projected/0c8b66b1-299f-4cae-93ba-cef113771543-kube-api-access-klsfx\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.445799 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445746 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c51a827b-a50d-479f-a6a2-e467ed632aae-kubelet-config\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.445799 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445792 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysconfig\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.445938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysctl-d\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.445938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445853 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysctl-conf\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.445938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.445917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-device-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.446201 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.446154 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.446275 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.446207 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7c63268-8eba-4619-9675-7ea82bcba390-etc-tuned\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.446275 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.446250 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsvng\" (UniqueName: \"kubernetes.io/projected/c7c63268-8eba-4619-9675-7ea82bcba390-kube-api-access-fsvng\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.446377 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.446285 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.446377 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.446320 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-systemd\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.446475 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.446395 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-run\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.446571 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.446540 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-sys\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.447176 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.447153 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-var-lib-kubelet\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.447890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.447871 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 19:27:26.447971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.447882 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 19:27:26.448116 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.447693 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 19:27:26.448478 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.448460 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 19:27:26.448570 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.448554 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 19:27:26.448781 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.448765 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jqw89\"" Jul 20 19:27:26.449212 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.449189 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 19:27:26.485004 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.484952 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 19:22:25 +0000 UTC" deadline="2028-04-04 08:12:00.70812018 +0000 UTC" Jul 20 19:27:26.485004 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.485004 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14964h44m34.223120643s" Jul 20 19:27:26.534390 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.534360 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 19:27:26.535941 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.535909 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" event={"ID":"5c31779b826e1e93da8d4838893e9654","Type":"ContainerStarted","Data":"c78b7580f881ed6a69b2cc1a4423bee4923b8b4a4f28063495a8910ddb0f8e3a"} Jul 20 19:27:26.547354 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.547470 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547363 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-cni-binary-copy\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.547470 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.547470 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.547470 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547455 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547492 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e7344064-0d35-4e15-abae-379facb8bff0-konnectivity-ca\") pod \"konnectivity-agent-vxjjl\" (UID: \"e7344064-0d35-4e15-abae-379facb8bff0\") " pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547518 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4a5474d-673d-433f-97b4-ad599865fa24-iptables-alerter-script\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547542 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-serviceca\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-cnibin\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547577 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-socket-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysctl-d\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.547626 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547630 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547661 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-cnibin\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.547698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547674 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsvng\" (UniqueName: \"kubernetes.io/projected/c7c63268-8eba-4619-9675-7ea82bcba390-kube-api-access-fsvng\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.547736 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret podName:c51a827b-a50d-479f-a6a2-e467ed632aae nodeName:}" failed. No retries permitted until 2026-07-20 19:27:27.047687161 +0000 UTC m=+3.030980095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret") pod "global-pull-secret-syncer-pw8fv" (UID: "c51a827b-a50d-479f-a6a2-e467ed632aae") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547791 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysctl-d\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-cni-multus\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547850 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0dc38f51-e754-436a-b690-3f07b631aff6-ovn-node-metrics-cert\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547885 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547911 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-env-overrides\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547937 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d00559d-ac72-41c0-ab2d-68879013c63a-hosts-file\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-run-netns\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.547994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548019 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-system-cni-dir\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548027 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0d00559d-ac72-41c0-ab2d-68879013c63a-hosts-file\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548045 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548071 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-kubernetes\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548093 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65bsd\" (UniqueName: \"kubernetes.io/projected/0d00559d-ac72-41c0-ab2d-68879013c63a-kube-api-access-65bsd\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.548197 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548179 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-kubernetes\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548211 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-cni-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548232 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-os-release\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548257 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-k8s-cni-cncf-io\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-hostroot\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548333 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-daemon-config\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548361 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmnrm\" (UniqueName: \"kubernetes.io/projected/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-kube-api-access-lmnrm\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548387 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-slash\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548418 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-var-lib-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548427 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e7344064-0d35-4e15-abae-379facb8bff0-konnectivity-ca\") pod \"konnectivity-agent-vxjjl\" (UID: \"e7344064-0d35-4e15-abae-379facb8bff0\") " pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548446 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a4a5474d-673d-433f-97b4-ad599865fa24-iptables-alerter-script\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548473 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548514 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-etc-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548535 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-serviceca\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548578 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-system-cni-dir\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548599 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d00559d-ac72-41c0-ab2d-68879013c63a-tmp-dir\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548752 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdvgp\" (UniqueName: \"kubernetes.io/projected/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-kube-api-access-tdvgp\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.548964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548735 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548809 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548841 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frdlf\" (UniqueName: \"kubernetes.io/projected/e25cc402-8092-42b7-89c9-54619d601351-kube-api-access-frdlf\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548866 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7c63268-8eba-4619-9675-7ea82bcba390-tmp\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548890 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e7344064-0d35-4e15-abae-379facb8bff0-agent-certs\") pod \"konnectivity-agent-vxjjl\" (UID: \"e7344064-0d35-4e15-abae-379facb8bff0\") " pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-kubelet\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548935 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-cni-netd\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548958 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d00559d-ac72-41c0-ab2d-68879013c63a-tmp-dir\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.548959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-system-cni-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549019 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-modprobe-d\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549043 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4a5474d-673d-433f-97b4-ad599865fa24-host-slash\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549089 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54jcv\" (UniqueName: \"kubernetes.io/projected/0dc38f51-e754-436a-b690-3f07b631aff6-kube-api-access-54jcv\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549089 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-registration-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549137 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549163 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klsfx\" (UniqueName: \"kubernetes.io/projected/0c8b66b1-299f-4cae-93ba-cef113771543-kube-api-access-klsfx\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549166 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a4a5474d-673d-433f-97b4-ad599865fa24-host-slash\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549179 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-modprobe-d\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.549760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549205 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-kubelet-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549212 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c51a827b-a50d-479f-a6a2-e467ed632aae-kubelet-config\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549204 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysconfig\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549287 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysctl-conf\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549308 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7c63268-8eba-4619-9675-7ea82bcba390-etc-tuned\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549322 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c51a827b-a50d-479f-a6a2-e467ed632aae-kubelet-config\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549325 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysconfig\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549331 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-log-socket\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549368 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-device-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549393 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-socket-dir-parent\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549421 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-sysctl-conf\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549418 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-multus-certs\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549456 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-device-dir\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549475 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549508 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-systemd\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-run\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-sys\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.550554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549565 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-sys-fs\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549641 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-var-lib-kubelet\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549674 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-cni-bin\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549685 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-sys\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549642 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-etc-systemd\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549697 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-systemd\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549687 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-run\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549719 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-ovn\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-lib-modules\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549744 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-var-lib-kubelet\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-host\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549828 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-netns\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549850 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-host\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-etc-kubernetes\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549871 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7c63268-8eba-4619-9675-7ea82bcba390-lib-modules\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549882 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-systemd-units\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.549979 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550055 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-cni-bin\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550111 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c51a827b-a50d-479f-a6a2-e467ed632aae-dbus\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550155 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-kubelet\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550186 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-conf-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550208 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-ovnkube-script-lib\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550265 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c51a827b-a50d-479f-a6a2-e467ed632aae-dbus\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550277 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-cni-binary-copy\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550344 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnrr4\" (UniqueName: \"kubernetes.io/projected/a4a5474d-673d-433f-97b4-ad599865fa24-kube-api-access-mnrr4\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550381 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-host\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550414 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-cnibin\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550440 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-node-log\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550485 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-ovnkube-config\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550526 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-host\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550574 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550637 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-os-release\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.551971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9nwn\" (UniqueName: \"kubernetes.io/projected/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-kube-api-access-w9nwn\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.552758 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550809 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0c8b66b1-299f-4cae-93ba-cef113771543-os-release\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.552758 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550905 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e25cc402-8092-42b7-89c9-54619d601351-etc-selinux\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.552758 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.550941 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0c8b66b1-299f-4cae-93ba-cef113771543-cni-binary-copy\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.553084 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.553061 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7c63268-8eba-4619-9675-7ea82bcba390-etc-tuned\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.553326 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.553308 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e7344064-0d35-4e15-abae-379facb8bff0-agent-certs\") pod \"konnectivity-agent-vxjjl\" (UID: \"e7344064-0d35-4e15-abae-379facb8bff0\") " pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.553508 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.553471 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7c63268-8eba-4619-9675-7ea82bcba390-tmp\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.555405 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.555386 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:26.555495 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.555410 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:26.555495 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.555426 2573 projected.go:196] Error preparing data for projected volume kube-api-access-8cnj7 for pod openshift-network-diagnostics/network-check-target-rcc5g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:26.555495 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.555480 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7 podName:2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:27.055462283 +0000 UTC m=+3.038755231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8cnj7" (UniqueName: "kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7") pod "network-check-target-rcc5g" (UID: "2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:26.557195 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.557163 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsvng\" (UniqueName: \"kubernetes.io/projected/c7c63268-8eba-4619-9675-7ea82bcba390-kube-api-access-fsvng\") pod \"tuned-tmqdj\" (UID: \"c7c63268-8eba-4619-9675-7ea82bcba390\") " pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.558566 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.558387 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65bsd\" (UniqueName: \"kubernetes.io/projected/0d00559d-ac72-41c0-ab2d-68879013c63a-kube-api-access-65bsd\") pod \"node-resolver-qxwxz\" (UID: \"0d00559d-ac72-41c0-ab2d-68879013c63a\") " pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.558566 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.558529 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frdlf\" (UniqueName: \"kubernetes.io/projected/e25cc402-8092-42b7-89c9-54619d601351-kube-api-access-frdlf\") pod \"aws-ebs-csi-driver-node-phz2q\" (UID: \"e25cc402-8092-42b7-89c9-54619d601351\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.559479 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.559438 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnrr4\" (UniqueName: \"kubernetes.io/projected/a4a5474d-673d-433f-97b4-ad599865fa24-kube-api-access-mnrr4\") pod \"iptables-alerter-h8qk9\" (UID: \"a4a5474d-673d-433f-97b4-ad599865fa24\") " pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.559971 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.559926 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9nwn\" (UniqueName: \"kubernetes.io/projected/d7dc7bed-e5e1-494e-af3e-3420e9712c1b-kube-api-access-w9nwn\") pod \"node-ca-ng274\" (UID: \"d7dc7bed-e5e1-494e-af3e-3420e9712c1b\") " pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.560495 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.560474 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klsfx\" (UniqueName: \"kubernetes.io/projected/0c8b66b1-299f-4cae-93ba-cef113771543-kube-api-access-klsfx\") pod \"multus-additional-cni-plugins-wd7w8\" (UID: \"0c8b66b1-299f-4cae-93ba-cef113771543\") " pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.651590 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651552 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-cni-binary-copy\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.651769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651628 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-cni-multus\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.651769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651657 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0dc38f51-e754-436a-b690-3f07b631aff6-ovn-node-metrics-cert\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.651769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-env-overrides\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.651769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651705 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-run-netns\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.651769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651728 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.651769 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651754 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-cni-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651777 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-os-release\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651778 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-cni-multus\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651801 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-k8s-cni-cncf-io\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651808 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-run-netns\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651847 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-k8s-cni-cncf-io\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651859 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-cni-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651884 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651892 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-hostroot\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-daemon-config\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651934 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-os-release\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651948 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmnrm\" (UniqueName: \"kubernetes.io/projected/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-kube-api-access-lmnrm\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651958 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-hostroot\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.651983 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-slash\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-var-lib-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652035 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-etc-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652060 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdvgp\" (UniqueName: \"kubernetes.io/projected/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-kube-api-access-tdvgp\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652083 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-var-lib-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-kubelet\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652127 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-slash\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-cni-netd\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652171 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-system-cni-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652197 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54jcv\" (UniqueName: \"kubernetes.io/projected/0dc38f51-e754-436a-b690-3f07b631aff6-kube-api-access-54jcv\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652196 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-cni-binary-copy\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652201 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-kubelet\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652228 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-log-socket\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652243 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-cni-netd\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652255 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-socket-dir-parent\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652276 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-system-cni-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652280 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-multus-certs\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652289 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-etc-openvswitch\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652323 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-cni-bin\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652326 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-multus-certs\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652355 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-log-socket\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652376 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-cni-bin\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.652869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652376 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-daemon-config\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652376 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-socket-dir-parent\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652460 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-systemd\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652487 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-ovn\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652497 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-systemd\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-netns\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652538 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-run-ovn\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-etc-kubernetes\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652593 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-run-netns\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652593 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-systemd-units\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652641 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-systemd-units\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652641 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652670 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-run-ovn-kubernetes\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652674 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-cni-bin\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652706 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-kubelet\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652707 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-etc-kubernetes\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652743 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-host-var-lib-kubelet\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652747 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-cni-bin\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.653695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-conf-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652796 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-multus-conf-dir\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652802 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-ovnkube-script-lib\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652862 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-cnibin\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.652868 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-node-log\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-ovnkube-config\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:26.652929 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:27:27.15291201 +0000 UTC m=+3.136204958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.652959 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.653023 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.653293 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-cnibin\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.653339 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0dc38f51-e754-436a-b690-3f07b631aff6-node-log\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.653366 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-env-overrides\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.653390 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-ovnkube-script-lib\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.653406 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0dc38f51-e754-436a-b690-3f07b631aff6-ovnkube-config\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.654444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.654233 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0dc38f51-e754-436a-b690-3f07b631aff6-ovn-node-metrics-cert\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.661418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.661388 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54jcv\" (UniqueName: \"kubernetes.io/projected/0dc38f51-e754-436a-b690-3f07b631aff6-kube-api-access-54jcv\") pod \"ovnkube-node-2vrt6\" (UID: \"0dc38f51-e754-436a-b690-3f07b631aff6\") " pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.661536 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.661518 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdvgp\" (UniqueName: \"kubernetes.io/projected/0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de-kube-api-access-tdvgp\") pod \"multus-gnnjj\" (UID: \"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de\") " pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.661580 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.661529 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmnrm\" (UniqueName: \"kubernetes.io/projected/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-kube-api-access-lmnrm\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:26.749893 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.749819 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" Jul 20 19:27:26.756501 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.756480 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:26.766085 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.766062 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" Jul 20 19:27:26.773693 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.773671 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h8qk9" Jul 20 19:27:26.781193 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.781173 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" Jul 20 19:27:26.789048 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.789029 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qxwxz" Jul 20 19:27:26.795592 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.795573 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ng274" Jul 20 19:27:26.803311 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.803292 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gnnjj" Jul 20 19:27:26.809003 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.808982 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:26.946826 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:26.946791 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 19:27:27.055927 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.055839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:27.055927 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.055904 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:27.056146 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.056007 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:27.056146 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.056033 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:27.056146 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.056059 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:27.056146 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.056067 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret podName:c51a827b-a50d-479f-a6a2-e467ed632aae nodeName:}" failed. No retries permitted until 2026-07-20 19:27:28.056048949 +0000 UTC m=+4.039341882 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret") pod "global-pull-secret-syncer-pw8fv" (UID: "c51a827b-a50d-479f-a6a2-e467ed632aae") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:27.056146 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.056072 2573 projected.go:196] Error preparing data for projected volume kube-api-access-8cnj7 for pod openshift-network-diagnostics/network-check-target-rcc5g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:27.056146 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.056125 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7 podName:2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:28.056109328 +0000 UTC m=+4.039402276 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cnj7" (UniqueName: "kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7") pod "network-check-target-rcc5g" (UID: "2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:27.156908 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.156880 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:27.157040 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.157014 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:27.157100 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:27.157067 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:27:28.157052971 +0000 UTC m=+4.140345908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:27.483760 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:27.483735 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc38f51_e754_436a_b690_3f07b631aff6.slice/crio-f5f8975e79d7c30ee8c738bb40ada6c34edd2c7835d9b6868e6403492ad57d3f WatchSource:0}: Error finding container f5f8975e79d7c30ee8c738bb40ada6c34edd2c7835d9b6868e6403492ad57d3f: Status 404 returned error can't find the container with id f5f8975e79d7c30ee8c738bb40ada6c34edd2c7835d9b6868e6403492ad57d3f Jul 20 19:27:27.485152 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.485101 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 19:22:25 +0000 UTC" deadline="2027-12-19 22:59:18.142208418 +0000 UTC" Jul 20 19:27:27.485236 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.485152 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12411h31m50.657059169s" Jul 20 19:27:27.488464 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:27.488222 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6c2f97_ab28_42fc_80c1_3e62bc3bb5de.slice/crio-ae7128879d828f8ae21e2a3cda70cfb00ba0d097d89a0b0c66089aa0c4cf356a WatchSource:0}: Error finding container ae7128879d828f8ae21e2a3cda70cfb00ba0d097d89a0b0c66089aa0c4cf356a: Status 404 returned error can't find the container with id ae7128879d828f8ae21e2a3cda70cfb00ba0d097d89a0b0c66089aa0c4cf356a Jul 20 19:27:27.490199 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:27.490162 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode25cc402_8092_42b7_89c9_54619d601351.slice/crio-afe1fcc6efab0da916ce0439f4db33e3cff46d818ca62a0938717a83b41f7b56 WatchSource:0}: Error finding container afe1fcc6efab0da916ce0439f4db33e3cff46d818ca62a0938717a83b41f7b56: Status 404 returned error can't find the container with id afe1fcc6efab0da916ce0439f4db33e3cff46d818ca62a0938717a83b41f7b56 Jul 20 19:27:27.491014 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:27.490908 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a5474d_673d_433f_97b4_ad599865fa24.slice/crio-609f5cbd71d2876b7e9d8f05101403ead3e7f0704bbe3a5021c46b485f4065a4 WatchSource:0}: Error finding container 609f5cbd71d2876b7e9d8f05101403ead3e7f0704bbe3a5021c46b485f4065a4: Status 404 returned error can't find the container with id 609f5cbd71d2876b7e9d8f05101403ead3e7f0704bbe3a5021c46b485f4065a4 Jul 20 19:27:27.492636 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:27.492405 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d00559d_ac72_41c0_ab2d_68879013c63a.slice/crio-38c0737f933bfd8d234a79bcc86af5499dcbc5e00fa0c450af0480a9303db85c WatchSource:0}: Error finding container 38c0737f933bfd8d234a79bcc86af5499dcbc5e00fa0c450af0480a9303db85c: Status 404 returned error can't find the container with id 38c0737f933bfd8d234a79bcc86af5499dcbc5e00fa0c450af0480a9303db85c Jul 20 19:27:27.540181 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.540153 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxjjl" event={"ID":"e7344064-0d35-4e15-abae-379facb8bff0","Type":"ContainerStarted","Data":"a8ac2e0fd9aae05b284324c8bb228f17ed0b4b6431988cb1ebac6c5085b3547e"} Jul 20 19:27:27.541222 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.541199 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" event={"ID":"c7c63268-8eba-4619-9675-7ea82bcba390","Type":"ContainerStarted","Data":"9173d4703aeadc7390e9e8645d800b4055ae5083bb51ab83d1bba4bbe2d926b2"} Jul 20 19:27:27.542104 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.542083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ng274" event={"ID":"d7dc7bed-e5e1-494e-af3e-3420e9712c1b","Type":"ContainerStarted","Data":"7f91551b3a744837fb7bc93608bf3fd5da4e3cb82a8d0d46767ca5a4c90e3161"} Jul 20 19:27:27.543063 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.543043 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qxwxz" event={"ID":"0d00559d-ac72-41c0-ab2d-68879013c63a","Type":"ContainerStarted","Data":"38c0737f933bfd8d234a79bcc86af5499dcbc5e00fa0c450af0480a9303db85c"} Jul 20 19:27:27.544050 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.544028 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gnnjj" event={"ID":"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de","Type":"ContainerStarted","Data":"ae7128879d828f8ae21e2a3cda70cfb00ba0d097d89a0b0c66089aa0c4cf356a"} Jul 20 19:27:27.544973 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.544956 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"f5f8975e79d7c30ee8c738bb40ada6c34edd2c7835d9b6868e6403492ad57d3f"} Jul 20 19:27:27.546141 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.546109 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerStarted","Data":"a72f4fbbdaa4f93d7f8c5e5272cb2ee7d407504e7249eedcffede6611ac49080"} Jul 20 19:27:27.547274 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.547230 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8qk9" event={"ID":"a4a5474d-673d-433f-97b4-ad599865fa24","Type":"ContainerStarted","Data":"609f5cbd71d2876b7e9d8f05101403ead3e7f0704bbe3a5021c46b485f4065a4"} Jul 20 19:27:27.548356 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:27.548332 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" event={"ID":"e25cc402-8092-42b7-89c9-54619d601351","Type":"ContainerStarted","Data":"afe1fcc6efab0da916ce0439f4db33e3cff46d818ca62a0938717a83b41f7b56"} Jul 20 19:27:28.061755 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.061502 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:28.061922 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.061821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:28.062006 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.061991 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:28.062069 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.062014 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:28.062069 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.062028 2573 projected.go:196] Error preparing data for projected volume kube-api-access-8cnj7 for pod openshift-network-diagnostics/network-check-target-rcc5g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:28.062169 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.062089 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7 podName:2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:30.062070767 +0000 UTC m=+6.045363717 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cnj7" (UniqueName: "kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7") pod "network-check-target-rcc5g" (UID: "2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:28.062583 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.062486 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:28.062583 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.062562 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret podName:c51a827b-a50d-479f-a6a2-e467ed632aae nodeName:}" failed. No retries permitted until 2026-07-20 19:27:30.062542426 +0000 UTC m=+6.045835381 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret") pod "global-pull-secret-syncer-pw8fv" (UID: "c51a827b-a50d-479f-a6a2-e467ed632aae") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:28.162396 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.162359 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:28.162590 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.162538 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:28.162719 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.162604 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:27:30.162584875 +0000 UTC m=+6.145877811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:28.533800 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.533769 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:28.534357 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.533896 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:28.534357 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.534285 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:28.534474 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.534398 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:28.534474 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.534456 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:28.534570 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:28.534520 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:28.569849 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.569815 2573 generic.go:356] "Generic (PLEG): container finished" podID="694a9c0ad5d708ffa615de056b509f7a" containerID="88577ecd61c59b34fcbe3a62b96e4519a5c795f5e475982c1d31e95a82b915e4" exitCode=0 Jul 20 19:27:28.569990 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.569904 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" event={"ID":"694a9c0ad5d708ffa615de056b509f7a","Type":"ContainerDied","Data":"88577ecd61c59b34fcbe3a62b96e4519a5c795f5e475982c1d31e95a82b915e4"} Jul 20 19:27:28.576127 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:28.575813 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" event={"ID":"5c31779b826e1e93da8d4838893e9654","Type":"ContainerStarted","Data":"cef30541ab58f7300f3fe691795e9aaf468c0576632e348c5d8b320eee12b7ca"} Jul 20 19:27:29.588730 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:29.588672 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" event={"ID":"694a9c0ad5d708ffa615de056b509f7a","Type":"ContainerStarted","Data":"35923ad7bcaf7570f1625e5be467093cf04b88081a2dbc37adeb0fad6abd89d7"} Jul 20 19:27:29.606543 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:29.606428 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-93.ec2.internal" podStartSLOduration=4.606406893 podStartE2EDuration="4.606406893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:28.600380696 +0000 UTC m=+4.583673654" watchObservedRunningTime="2026-07-20 19:27:29.606406893 +0000 UTC m=+5.589699850" Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:30.076921 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:30.076991 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.077152 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.077209 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret podName:c51a827b-a50d-479f-a6a2-e467ed632aae nodeName:}" failed. No retries permitted until 2026-07-20 19:27:34.077190606 +0000 UTC m=+10.060483542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret") pod "global-pull-secret-syncer-pw8fv" (UID: "c51a827b-a50d-479f-a6a2-e467ed632aae") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.077579 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.077601 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.077630 2573 projected.go:196] Error preparing data for projected volume kube-api-access-8cnj7 for pod openshift-network-diagnostics/network-check-target-rcc5g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:30.077698 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.077673 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7 podName:2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:34.077659513 +0000 UTC m=+10.060952449 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cnj7" (UniqueName: "kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7") pod "network-check-target-rcc5g" (UID: "2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:30.178561 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:30.177779 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:30.178561 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.177968 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:30.178561 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.178028 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:27:34.178010845 +0000 UTC m=+10.161303782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:30.532397 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:30.531703 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:30.532397 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.531848 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:30.532397 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:30.532258 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:30.532397 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.532350 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:30.533021 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:30.532834 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:30.533021 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:30.532941 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:32.530805 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:32.530653 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:32.530805 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:32.530788 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:32.531286 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:32.531173 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:32.531332 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:32.531286 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:32.531385 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:32.531366 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:32.531462 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:32.531440 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:34.112268 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:34.112231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:34.112757 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:34.112302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:34.112757 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.112420 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:34.112757 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.112479 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret podName:c51a827b-a50d-479f-a6a2-e467ed632aae nodeName:}" failed. No retries permitted until 2026-07-20 19:27:42.112459332 +0000 UTC m=+18.095752274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret") pod "global-pull-secret-syncer-pw8fv" (UID: "c51a827b-a50d-479f-a6a2-e467ed632aae") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:34.113063 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.112783 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:34.113063 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.112814 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:34.113063 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.112827 2573 projected.go:196] Error preparing data for projected volume kube-api-access-8cnj7 for pod openshift-network-diagnostics/network-check-target-rcc5g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:34.113063 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.112883 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7 podName:2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:42.112865843 +0000 UTC m=+18.096158779 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cnj7" (UniqueName: "kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7") pod "network-check-target-rcc5g" (UID: "2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:34.213412 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:34.213372 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:34.213599 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.213538 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:34.213688 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.213629 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:27:42.213592688 +0000 UTC m=+18.196885625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:34.531726 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:34.531690 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:34.531889 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.531821 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:34.531960 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:34.531889 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:34.532016 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.531957 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:34.532311 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:34.532294 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:34.532413 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:34.532393 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:36.531074 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:36.531038 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:36.531512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:36.531037 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:36.531512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:36.531167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:36.531512 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:36.531165 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:36.531512 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:36.531255 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:36.531512 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:36.531351 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:38.530954 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:38.530920 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:38.531395 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:38.530916 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:38.531395 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:38.531037 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:38.531395 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:38.530922 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:38.531395 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:38.531122 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:38.531395 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:38.531222 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:40.534061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:40.531304 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:40.534061 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:40.531503 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:40.534061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:40.531638 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:40.534061 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:40.531780 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:40.534061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:40.531810 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:40.534061 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:40.531945 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:42.175816 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:42.175780 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:42.176271 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:42.175839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:42.176271 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.175962 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:42.176271 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.176018 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret podName:c51a827b-a50d-479f-a6a2-e467ed632aae nodeName:}" failed. No retries permitted until 2026-07-20 19:27:58.176004014 +0000 UTC m=+34.159296946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret") pod "global-pull-secret-syncer-pw8fv" (UID: "c51a827b-a50d-479f-a6a2-e467ed632aae") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:42.176271 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.175964 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:42.176271 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.176049 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:42.176271 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.176061 2573 projected.go:196] Error preparing data for projected volume kube-api-access-8cnj7 for pod openshift-network-diagnostics/network-check-target-rcc5g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:42.176271 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.176098 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7 podName:2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:58.176088367 +0000 UTC m=+34.159381304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cnj7" (UniqueName: "kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7") pod "network-check-target-rcc5g" (UID: "2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:42.276887 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:42.276851 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:42.277050 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.276970 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:42.277119 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.277060 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:27:58.277038729 +0000 UTC m=+34.260331687 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:42.531238 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:42.531154 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:42.531393 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:42.531154 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:42.531393 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.531272 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:42.531393 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.531329 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:42.531393 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:42.531154 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:42.531605 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:42.531416 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:44.531084 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:44.531051 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:44.531532 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:44.531167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:44.531532 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:44.531240 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:44.531532 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:44.531160 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:44.531532 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:44.531269 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:44.531532 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:44.531387 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:46.530853 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.530553 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:46.531514 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.530553 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:46.531514 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:46.530928 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:46.531514 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:46.531002 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:46.531514 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.530648 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:46.531514 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:46.531063 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:46.631194 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.631148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxjjl" event={"ID":"e7344064-0d35-4e15-abae-379facb8bff0","Type":"ContainerStarted","Data":"1bd26eab7668b1b3b7fce7cf66f8ecfef293c91f1a16b99067d6d458a0274688"} Jul 20 19:27:46.632698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.632673 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" event={"ID":"c7c63268-8eba-4619-9675-7ea82bcba390","Type":"ContainerStarted","Data":"644155c70463ee986b22f50785bbadc645867d4de4ee61aefa8e7d91ac1bb16e"} Jul 20 19:27:46.634027 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.634004 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ng274" event={"ID":"d7dc7bed-e5e1-494e-af3e-3420e9712c1b","Type":"ContainerStarted","Data":"c05a0cc24a91a52a5daf45b140bf42b602708d8dcf87673bbfd65d8e71e934d9"} Jul 20 19:27:46.635371 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.635337 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qxwxz" event={"ID":"0d00559d-ac72-41c0-ab2d-68879013c63a","Type":"ContainerStarted","Data":"e1ae777ba72186307d8ae309b5cf8ce511cbdd9804118d61e8b6afa2dc729f4e"} Jul 20 19:27:46.636651 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.636630 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gnnjj" event={"ID":"0f6c2f97-ab28-42fc-80c1-3e62bc3bb5de","Type":"ContainerStarted","Data":"6a8c40bb620709fe2ab727c83f6606e80b8ad84f0fa23f649c04f5b1b341c91e"} Jul 20 19:27:46.639365 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.639346 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"cddea8f107f532a9316cc7986825345a930edf27fd05acc1b9119261d38bd216"} Jul 20 19:27:46.639442 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.639416 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"792f04ee2660f7309a41b18a75958277e5daab59e628e72382faf140176ab7c7"} Jul 20 19:27:46.639492 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.639467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"c1507dc5a9c159fdbf748cf15cc232835111e535e6c9b04acc2732afc4180a17"} Jul 20 19:27:46.639542 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.639514 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"ac2a7f0c5db78617cf8953877e634abac398b12eff0934b2761544fb86fe4d44"} Jul 20 19:27:46.639584 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.639554 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"3f61db4103270da59655d31d4136b08ec37ccda29c472994e46e842d6b60bc99"} Jul 20 19:27:46.639584 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.639582 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"729a664c3c7df25ceb016744ede53b6e386919e829b42b9940a38fa5591ab336"} Jul 20 19:27:46.640845 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.640824 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c8b66b1-299f-4cae-93ba-cef113771543" containerID="7ba518067d3c0765042cc428cf950eba99bee5f4ec82c10c5e8a33d393dcd2e4" exitCode=0 Jul 20 19:27:46.640921 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.640900 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerDied","Data":"7ba518067d3c0765042cc428cf950eba99bee5f4ec82c10c5e8a33d393dcd2e4"} Jul 20 19:27:46.642435 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.642415 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" event={"ID":"e25cc402-8092-42b7-89c9-54619d601351","Type":"ContainerStarted","Data":"8b7ae05afd79982da339a4b65f8c3cc5f7c7fd4d3f82911f538f1890790688c3"} Jul 20 19:27:46.645143 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.645106 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-93.ec2.internal" podStartSLOduration=21.645094647 podStartE2EDuration="21.645094647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:29.605966779 +0000 UTC m=+5.589259758" watchObservedRunningTime="2026-07-20 19:27:46.645094647 +0000 UTC m=+22.628387602" Jul 20 19:27:46.645474 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.645446 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vxjjl" podStartSLOduration=8.49427348 podStartE2EDuration="22.645436338s" totalImagesPullingTime="14.151162858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:46.644699436 +0000 UTC m=+22.627992411" watchObservedRunningTime="2026-07-20 19:27:46.645436338 +0000 UTC m=+22.628729292" Jul 20 19:27:46.658446 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.658409 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qxwxz" podStartSLOduration=4.223154559 podStartE2EDuration="22.658398141s" totalImagesPullingTime="18.435243582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:46.658146435 +0000 UTC m=+22.641439391" watchObservedRunningTime="2026-07-20 19:27:46.658398141 +0000 UTC m=+22.641691095" Jul 20 19:27:46.671674 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.671640 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ng274" podStartSLOduration=4.239427648 podStartE2EDuration="22.671630066s" totalImagesPullingTime="18.432202418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:46.671112215 +0000 UTC m=+22.654405161" watchObservedRunningTime="2026-07-20 19:27:46.671630066 +0000 UTC m=+22.654923012" Jul 20 19:27:46.687379 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.687347 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gnnjj" podStartSLOduration=4.190467545 podStartE2EDuration="22.687337345s" totalImagesPullingTime="18.4968698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:46.687217933 +0000 UTC m=+22.670510887" watchObservedRunningTime="2026-07-20 19:27:46.687337345 +0000 UTC m=+22.670630300" Jul 20 19:27:46.728066 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:46.727990 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tmqdj" podStartSLOduration=4.295761264 podStartE2EDuration="22.727978151s" totalImagesPullingTime="18.432216887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:46.727730872 +0000 UTC m=+22.711023828" watchObservedRunningTime="2026-07-20 19:27:46.727978151 +0000 UTC m=+22.711271105" Jul 20 19:27:47.505913 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:47.505890 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 19:27:47.646733 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:47.646642 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" event={"ID":"e25cc402-8092-42b7-89c9-54619d601351","Type":"ContainerStarted","Data":"0cc0d56b6808dd1ada77f68c884b294b429107424b9ed9cb55ff33636327dc41"} Jul 20 19:27:47.648045 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:47.648023 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h8qk9" event={"ID":"a4a5474d-673d-433f-97b4-ad599865fa24","Type":"ContainerStarted","Data":"1c7378e8805331ce80d48a60da39a095acebc4c3503c60a1a7f4a86290f3e79a"} Jul 20 19:27:47.663423 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:47.663308 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h8qk9" podStartSLOduration=5.227594228 podStartE2EDuration="23.663289992s" totalImagesPullingTime="18.435695764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:47.662509189 +0000 UTC m=+23.645802144" watchObservedRunningTime="2026-07-20 19:27:47.663289992 +0000 UTC m=+23.646582949" Jul 20 19:27:47.949960 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:47.949875 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:47.950506 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:47.950484 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:48.492247 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.492135 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T19:27:47.505908531Z","UUID":"49565bac-e3c1-4749-9daa-04d524446cb7","Handler":null,"Name":"","Endpoint":""} Jul 20 19:27:48.493716 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.493687 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 19:27:48.493852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.493733 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 19:27:48.530628 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.530590 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:48.530628 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.530602 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:48.530842 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:48.530729 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:48.530842 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:48.530794 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:48.530949 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.530845 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:48.530999 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:48.530950 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:48.651602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.651573 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:48.652190 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:48.652166 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vxjjl" Jul 20 19:27:49.657999 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:49.657961 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"ee8cea8c9b46260866ca085ebbe39c1401cd39ff579eaf5ec59efc5f5b273dbe"} Jul 20 19:27:49.660672 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:49.660642 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" event={"ID":"e25cc402-8092-42b7-89c9-54619d601351","Type":"ContainerStarted","Data":"1f8cce61d400d9e74c6d95296ff423f154f4374df53b33132b93a73fee4dbe3e"} Jul 20 19:27:49.679530 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:49.679477 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-phz2q" podStartSLOduration=4.430969344 podStartE2EDuration="25.679457596s" totalImagesPullingTime="21.248488252s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:49.678895802 +0000 UTC m=+25.662188758" watchObservedRunningTime="2026-07-20 19:27:49.679457596 +0000 UTC m=+25.662750553" Jul 20 19:27:50.531583 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:50.531396 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:50.531781 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:50.531396 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:50.531781 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:50.531685 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:50.531781 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:50.531396 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:50.531781 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:50.531764 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:50.531994 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:50.531809 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:52.530701 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.530606 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:52.530701 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.530655 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:52.531399 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.530608 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:52.531399 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:52.530730 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:52.531399 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:52.530789 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:52.531399 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:52.530864 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:52.670307 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.670108 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" event={"ID":"0dc38f51-e754-436a-b690-3f07b631aff6","Type":"ContainerStarted","Data":"be17ff46d91b3a76f53672a1ec57ee33ecf633b81989b6f3864aec0eaaaeb395"} Jul 20 19:27:52.670490 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.670361 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:52.670490 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.670411 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:52.672075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.672042 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c8b66b1-299f-4cae-93ba-cef113771543" containerID="43c3a5b3d818aed3e351539b8ea734421cf23835559e98190aecaca5233e6436" exitCode=0 Jul 20 19:27:52.672186 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.672088 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerDied","Data":"43c3a5b3d818aed3e351539b8ea734421cf23835559e98190aecaca5233e6436"} Jul 20 19:27:52.685668 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.685648 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:52.705182 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:52.704889 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" podStartSLOduration=10.18139377 podStartE2EDuration="28.704870086s" totalImagesPullingTime="18.523476316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:27:52.703361591 +0000 UTC m=+28.686654547" watchObservedRunningTime="2026-07-20 19:27:52.704870086 +0000 UTC m=+28.688163044" Jul 20 19:27:53.675576 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:53.675546 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:53.690352 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:53.690327 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:27:54.285851 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.285774 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pw8fv"] Jul 20 19:27:54.285981 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.285880 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:54.285981 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:54.285952 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:54.290247 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.290219 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qndtv"] Jul 20 19:27:54.290370 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.290341 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:54.290479 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:54.290451 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:54.290869 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.290849 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rcc5g"] Jul 20 19:27:54.290974 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.290948 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:54.291054 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:54.291037 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:54.678763 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.678728 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c8b66b1-299f-4cae-93ba-cef113771543" containerID="0629fa2e73c38cab372a3c9980e54359b14dce5a642ff81d60dfcffd5a1a91d4" exitCode=0 Jul 20 19:27:54.679115 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:54.678801 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerDied","Data":"0629fa2e73c38cab372a3c9980e54359b14dce5a642ff81d60dfcffd5a1a91d4"} Jul 20 19:27:55.530896 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:55.530865 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:55.530896 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:55.530891 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:55.531108 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:55.530989 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:55.531165 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:55.531112 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:56.530665 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:56.530438 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:56.531089 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:56.530712 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:56.685750 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:56.685715 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c8b66b1-299f-4cae-93ba-cef113771543" containerID="6d3e60817a7f99cccc17607a58e30911102bebcb28278b357eafb68d323d65d9" exitCode=0 Jul 20 19:27:56.685901 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:56.685761 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerDied","Data":"6d3e60817a7f99cccc17607a58e30911102bebcb28278b357eafb68d323d65d9"} Jul 20 19:27:57.531173 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:57.531144 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:57.531515 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:57.531142 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:57.531515 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:57.531262 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pw8fv" podUID="c51a827b-a50d-479f-a6a2-e467ed632aae" Jul 20 19:27:57.531515 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:57.531312 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rcc5g" podUID="2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65" Jul 20 19:27:58.200580 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.200538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:58.200770 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.200602 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:58.200770 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.200707 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:58.200770 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.200715 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 19:27:58.200770 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.200737 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 19:27:58.200770 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.200750 2573 projected.go:196] Error preparing data for projected volume kube-api-access-8cnj7 for pod openshift-network-diagnostics/network-check-target-rcc5g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:58.200770 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.200767 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret podName:c51a827b-a50d-479f-a6a2-e467ed632aae nodeName:}" failed. No retries permitted until 2026-07-20 19:28:30.200747985 +0000 UTC m=+66.184040919 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret") pod "global-pull-secret-syncer-pw8fv" (UID: "c51a827b-a50d-479f-a6a2-e467ed632aae") : object "kube-system"/"original-pull-secret" not registered Jul 20 19:27:58.201060 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.200790 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7 podName:2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:30.200777854 +0000 UTC m=+66.184070787 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8cnj7" (UniqueName: "kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7") pod "network-check-target-rcc5g" (UID: "2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 19:27:58.301472 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.301442 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:58.301662 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.301557 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:58.301662 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.301635 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:28:30.301602766 +0000 UTC m=+66.284895703 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 19:27:58.530522 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.530450 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:27:58.530683 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:58.530580 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:27:58.858331 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.858247 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-93.ec2.internal" event="NodeReady" Jul 20 19:27:58.858783 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.858401 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 19:27:58.904514 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.904350 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl"] Jul 20 19:27:58.906413 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.906395 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b54cbdc8b-htc59"] Jul 20 19:27:58.906533 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.906512 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:27:58.908319 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.908291 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:58.908964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.908931 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dn7g9\"" Jul 20 19:27:58.908964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.908958 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 20 19:27:58.909243 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.909224 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 20 19:27:58.910303 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.910284 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 19:27:58.911425 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.911406 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dpg8d\"" Jul 20 19:27:58.911657 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.911639 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 19:27:58.911902 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.911831 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 19:27:58.916947 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.916925 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 19:27:58.919027 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.918995 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6gzpv"] Jul 20 19:27:58.922227 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.922207 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x2c5n"] Jul 20 19:27:58.922320 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.922271 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:58.924576 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.924558 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 19:27:58.925107 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.925085 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl"] Jul 20 19:27:58.925191 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.925140 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b54cbdc8b-htc59"] Jul 20 19:27:58.925251 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.925192 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:58.925658 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.925387 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27sg8\"" Jul 20 19:27:58.926024 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.926007 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 19:27:58.928681 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.928659 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 19:27:58.928811 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.928794 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 19:27:58.929000 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.928984 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-94xxz\"" Jul 20 19:27:58.929777 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.929759 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 19:27:58.937918 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.937896 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6gzpv"] Jul 20 19:27:58.943986 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:58.943968 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x2c5n"] Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.005939 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l4zd\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-kube-api-access-8l4zd\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-image-registry-private-configuration\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006212 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-trusted-ca\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006292 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-bound-sa-token\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-installation-pull-secrets\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006387 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006421 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-ca-trust-extracted\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006457 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb64f499-8e30-48b7-8bbd-7312d893df9e-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006487 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.008449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.006519 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-certificates\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.091863 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.091813 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vxjjl_e7344064-0d35-4e15-abae-379facb8bff0/konnectivity-agent/0.log" limit=1024 Jul 20 19:27:59.107599 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-installation-pull-secrets\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.107776 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107607 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/704c4f1f-46d5-4fc5-80ed-127d316b940b-cert\") pod \"ingress-canary-x2c5n\" (UID: \"704c4f1f-46d5-4fc5-80ed-127d316b940b\") " pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:59.107776 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107758 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:27:59.107890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107788 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-ca-trust-extracted\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.107890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107809 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.107890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107839 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb64f499-8e30-48b7-8bbd-7312d893df9e-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:27:59.107890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107863 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpcwt\" (UniqueName: \"kubernetes.io/projected/704c4f1f-46d5-4fc5-80ed-127d316b940b-kube-api-access-rpcwt\") pod \"ingress-canary-x2c5n\" (UID: \"704c4f1f-46d5-4fc5-80ed-127d316b940b\") " pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:59.107890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.108153 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107918 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-certificates\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.108153 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.107941 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45325d1-bac5-4328-abc6-3faa098afeb1-tmp-dir\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.108153 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.107940 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:27:59.108153 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108032 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l4zd\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-kube-api-access-8l4zd\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.108153 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108066 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-image-registry-private-configuration\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.108153 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108090 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-trusted-ca\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.108153 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108152 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-bound-sa-token\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.108447 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108178 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d45325d1-bac5-4328-abc6-3faa098afeb1-config-volume\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.108447 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108216 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-ca-trust-extracted\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.108447 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.108226 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:27:59.608199782 +0000 UTC m=+35.591492724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:27:59.108447 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-862s8\" (UniqueName: \"kubernetes.io/projected/d45325d1-bac5-4328-abc6-3faa098afeb1-kube-api-access-862s8\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.108708 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.108668 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:27:59.108708 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108677 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cb64f499-8e30-48b7-8bbd-7312d893df9e-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:27:59.108708 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.108683 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:27:59.108833 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.108761 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:59.608743384 +0000 UTC m=+35.592036325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:27:59.109092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.108969 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-certificates\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.109179 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.109154 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-trusted-ca\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.112216 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.112188 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-image-registry-private-configuration\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.112335 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.112222 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-installation-pull-secrets\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.116546 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.116511 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-bound-sa-token\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.116684 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.116598 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l4zd\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-kube-api-access-8l4zd\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.182344 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.182308 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vxjjl_e7344064-0d35-4e15-abae-379facb8bff0/konnectivity-agent/0.log" limit=1024 Jul 20 19:27:59.209307 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.209276 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.209452 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.209320 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpcwt\" (UniqueName: \"kubernetes.io/projected/704c4f1f-46d5-4fc5-80ed-127d316b940b-kube-api-access-rpcwt\") pod \"ingress-canary-x2c5n\" (UID: \"704c4f1f-46d5-4fc5-80ed-127d316b940b\") " pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:59.209452 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.209362 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45325d1-bac5-4328-abc6-3faa098afeb1-tmp-dir\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.209452 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.209400 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:27:59.209452 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.209426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d45325d1-bac5-4328-abc6-3faa098afeb1-config-volume\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.209452 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.209451 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-862s8\" (UniqueName: \"kubernetes.io/projected/d45325d1-bac5-4328-abc6-3faa098afeb1-kube-api-access-862s8\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.209734 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.209470 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:27:59.709451251 +0000 UTC m=+35.692744199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:27:59.209734 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.209507 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/704c4f1f-46d5-4fc5-80ed-127d316b940b-cert\") pod \"ingress-canary-x2c5n\" (UID: \"704c4f1f-46d5-4fc5-80ed-127d316b940b\") " pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:59.209939 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.209913 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d45325d1-bac5-4328-abc6-3faa098afeb1-tmp-dir\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.210106 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.210064 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d45325d1-bac5-4328-abc6-3faa098afeb1-config-volume\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.212128 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.212108 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/704c4f1f-46d5-4fc5-80ed-127d316b940b-cert\") pod \"ingress-canary-x2c5n\" (UID: \"704c4f1f-46d5-4fc5-80ed-127d316b940b\") " pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:59.219406 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.219368 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-862s8\" (UniqueName: \"kubernetes.io/projected/d45325d1-bac5-4328-abc6-3faa098afeb1-kube-api-access-862s8\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.219599 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.219578 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpcwt\" (UniqueName: \"kubernetes.io/projected/704c4f1f-46d5-4fc5-80ed-127d316b940b-kube-api-access-rpcwt\") pod \"ingress-canary-x2c5n\" (UID: \"704c4f1f-46d5-4fc5-80ed-127d316b940b\") " pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:59.243701 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.243673 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x2c5n" Jul 20 19:27:59.383533 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.383500 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x2c5n"] Jul 20 19:27:59.387899 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:27:59.387870 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704c4f1f_46d5_4fc5_80ed_127d316b940b.slice/crio-9e3fc5ea4e463ecb899d570a3700d79da6917d27721ad61f1a9430d68145828c WatchSource:0}: Error finding container 9e3fc5ea4e463ecb899d570a3700d79da6917d27721ad61f1a9430d68145828c: Status 404 returned error can't find the container with id 9e3fc5ea4e463ecb899d570a3700d79da6917d27721ad61f1a9430d68145828c Jul 20 19:27:59.531400 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.531367 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:27:59.531564 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.531544 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:27:59.533751 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.533724 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 19:27:59.533955 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.533862 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 19:27:59.534034 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.533964 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqcxh\"" Jul 20 19:27:59.534089 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.534051 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 19:27:59.614056 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.614017 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:27:59.614205 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.614120 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:27:59.614205 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.614173 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:27:59.614205 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.614195 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:27:59.614331 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.614249 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:27:59.614331 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.614258 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:00.614238482 +0000 UTC m=+36.597531424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:27:59.614331 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.614290 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:28:00.614277241 +0000 UTC m=+36.597570175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:27:59.693816 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.693730 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x2c5n" event={"ID":"704c4f1f-46d5-4fc5-80ed-127d316b940b","Type":"ContainerStarted","Data":"9e3fc5ea4e463ecb899d570a3700d79da6917d27721ad61f1a9430d68145828c"} Jul 20 19:27:59.714714 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:27:59.714683 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:27:59.714866 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.714846 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:27:59.714932 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:27:59.714918 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:00.714898781 +0000 UTC m=+36.698191729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:28:00.531094 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:00.530870 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:28:00.533242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:00.533221 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 19:28:00.533242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:00.533222 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pw7wf\"" Jul 20 19:28:00.622384 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:00.622347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:28:00.622570 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:00.622416 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:28:00.622570 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:00.622518 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:28:00.622719 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:00.622575 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:28:00.622719 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:00.622594 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:28:02.622572365 +0000 UTC m=+38.605865321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:28:00.622719 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:00.622594 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:28:00.622719 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:00.622647 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:02.622639917 +0000 UTC m=+38.605932854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:28:00.723816 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:00.723782 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:28:00.723979 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:00.723948 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:00.724037 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:00.724026 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:02.72400451 +0000 UTC m=+38.707297459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:28:02.320887 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:02.320844 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vxjjl_e7344064-0d35-4e15-abae-379facb8bff0/konnectivity-agent/0.log" limit=1024 Jul 20 19:28:02.416075 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:02.416043 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vxjjl_e7344064-0d35-4e15-abae-379facb8bff0/konnectivity-agent/0.log" limit=1024 Jul 20 19:28:02.639280 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:02.639239 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:28:02.639450 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:02.639298 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:28:02.639450 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:02.639388 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:28:02.639450 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:02.639398 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:28:02.639450 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:02.639411 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:28:02.639450 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:02.639449 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:06.639434875 +0000 UTC m=+42.622727813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:28:02.639674 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:02.639478 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:28:06.639459422 +0000 UTC m=+42.622752375 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:28:02.739808 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:02.739775 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:28:02.739952 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:02.739927 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:02.740015 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:02.740003 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:06.739987968 +0000 UTC m=+42.723280905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:28:03.705795 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:03.705765 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c8b66b1-299f-4cae-93ba-cef113771543" containerID="a0addf6424ae5450fc04c95f0f773794331001a5784c4e0ab63a676a1f0ba17b" exitCode=0 Jul 20 19:28:03.706291 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:03.705846 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerDied","Data":"a0addf6424ae5450fc04c95f0f773794331001a5784c4e0ab63a676a1f0ba17b"} Jul 20 19:28:03.707240 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:03.707221 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x2c5n" event={"ID":"704c4f1f-46d5-4fc5-80ed-127d316b940b","Type":"ContainerStarted","Data":"b1c141bfdc32dd1ca08dd97f6dd15d6be03b27829118ddb717f596e9535d6f39"} Jul 20 19:28:03.745228 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:03.745175 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x2c5n" podStartSLOduration=2.318404109 podStartE2EDuration="5.745158611s" totalImagesPullingTime="3.426754502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:03.74404002 +0000 UTC m=+39.727332975" watchObservedRunningTime="2026-07-20 19:28:03.745158611 +0000 UTC m=+39.728451567" Jul 20 19:28:04.712534 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:04.712498 2573 generic.go:356] "Generic (PLEG): container finished" podID="0c8b66b1-299f-4cae-93ba-cef113771543" containerID="788114d8fbf4dc4ccfc6da26caf131a237286cf976b55db1ec4e680b36ca6562" exitCode=0 Jul 20 19:28:04.713001 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:04.712588 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerDied","Data":"788114d8fbf4dc4ccfc6da26caf131a237286cf976b55db1ec4e680b36ca6562"} Jul 20 19:28:05.718227 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:05.718193 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" event={"ID":"0c8b66b1-299f-4cae-93ba-cef113771543","Type":"ContainerStarted","Data":"52cc68a342579e0ed8c48c5c66ad08a114163f5a76ad2b17d5468367c0232ab1"} Jul 20 19:28:05.739370 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:05.739323 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wd7w8" podStartSLOduration=9.408373417 podStartE2EDuration="41.739307218s" totalImagesPullingTime="32.330933801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:05.738501023 +0000 UTC m=+41.721793978" watchObservedRunningTime="2026-07-20 19:28:05.739307218 +0000 UTC m=+41.722600191" Jul 20 19:28:06.670627 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:06.670574 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:28:06.670799 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:06.670662 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:28:06.670799 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:06.670728 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:28:06.670799 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:06.670795 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:28:14.670779995 +0000 UTC m=+50.654072932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:28:06.670901 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:06.670801 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:28:06.670901 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:06.670823 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:28:06.670901 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:06.670867 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:14.670853716 +0000 UTC m=+50.654146648 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:28:06.770987 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:06.770957 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:28:06.771357 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:06.771107 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:06.771357 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:06.771170 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:14.771152616 +0000 UTC m=+50.754445550 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:28:14.720210 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:14.720170 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:28:14.720607 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:14.720236 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:28:14.720607 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:14.720318 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:28:14.720607 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:14.720382 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:28:30.720367303 +0000 UTC m=+66.703660251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:28:14.720607 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:14.720401 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:28:14.720607 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:14.720422 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:28:14.720607 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:14.720486 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:30.720470927 +0000 UTC m=+66.703763873 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:28:14.820847 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:14.820816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:28:14.821005 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:14.820924 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:14.821005 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:14.820980 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:28:30.820967008 +0000 UTC m=+66.804259941 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:28:25.696310 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:25.696279 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2vrt6" Jul 20 19:28:30.228461 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.228418 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:28:30.228461 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.228472 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:28:30.230853 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.230834 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 19:28:30.230916 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.230898 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 19:28:30.240993 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.240975 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 19:28:30.242584 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.242561 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c51a827b-a50d-479f-a6a2-e467ed632aae-original-pull-secret\") pod \"global-pull-secret-syncer-pw8fv\" (UID: \"c51a827b-a50d-479f-a6a2-e467ed632aae\") " pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:28:30.251689 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.251662 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cnj7\" (UniqueName: \"kubernetes.io/projected/2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65-kube-api-access-8cnj7\") pod \"network-check-target-rcc5g\" (UID: \"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65\") " pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:28:30.328780 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.328755 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:28:30.330739 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.330720 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 19:28:30.339721 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.339706 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 19:28:30.339771 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.339757 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:29:34.339741255 +0000 UTC m=+130.323034191 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : secret "metrics-daemon-secret" not found Jul 20 19:28:30.443988 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.443958 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pw8fv" Jul 20 19:28:30.453249 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.453225 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jqcxh\"" Jul 20 19:28:30.461784 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.461760 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:28:30.574982 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.574953 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pw8fv"] Jul 20 19:28:30.577875 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:28:30.577851 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc51a827b_a50d_479f_a6a2_e467ed632aae.slice/crio-b11c38281dcf3c1395395f4de55ac9be2f78dfeee296ad95e01b8d7f08b3177d WatchSource:0}: Error finding container b11c38281dcf3c1395395f4de55ac9be2f78dfeee296ad95e01b8d7f08b3177d: Status 404 returned error can't find the container with id b11c38281dcf3c1395395f4de55ac9be2f78dfeee296ad95e01b8d7f08b3177d Jul 20 19:28:30.589814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.589794 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rcc5g"] Jul 20 19:28:30.592829 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:28:30.592802 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f29c3d6_ae11_4e4e_b8a1_bf12cdc3ea65.slice/crio-0dd73bd0d6a44f850e38a5b1f310672e6940567d4af1af7d845688ba4e6d9e83 WatchSource:0}: Error finding container 0dd73bd0d6a44f850e38a5b1f310672e6940567d4af1af7d845688ba4e6d9e83: Status 404 returned error can't find the container with id 0dd73bd0d6a44f850e38a5b1f310672e6940567d4af1af7d845688ba4e6d9e83 Jul 20 19:28:30.732140 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.732100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:28:30.732315 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.732151 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:28:30.732315 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.732245 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:28:30.732315 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.732310 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:29:02.732293031 +0000 UTC m=+98.715585964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:28:30.732451 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.732249 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:28:30.732451 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.732344 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:28:30.732451 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.732387 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:29:02.732373934 +0000 UTC m=+98.715666867 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:28:30.774776 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.774688 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rcc5g" event={"ID":"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65","Type":"ContainerStarted","Data":"0dd73bd0d6a44f850e38a5b1f310672e6940567d4af1af7d845688ba4e6d9e83"} Jul 20 19:28:30.775672 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.775647 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pw8fv" event={"ID":"c51a827b-a50d-479f-a6a2-e467ed632aae","Type":"ContainerStarted","Data":"b11c38281dcf3c1395395f4de55ac9be2f78dfeee296ad95e01b8d7f08b3177d"} Jul 20 19:28:30.833181 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:30.833145 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:28:30.833315 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.833293 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:28:30.833364 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:28:30.833354 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:29:02.83333759 +0000 UTC m=+98.816630533 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:28:35.789991 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:35.789958 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rcc5g" event={"ID":"2f29c3d6-ae11-4e4e-b8a1-bf12cdc3ea65","Type":"ContainerStarted","Data":"e953216ff0b52113e3d9bcf7e79a1d9894883a300ae58a05b68f5538fe8a94cf"} Jul 20 19:28:35.790453 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:35.790046 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:28:35.791483 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:35.791453 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pw8fv" event={"ID":"c51a827b-a50d-479f-a6a2-e467ed632aae","Type":"ContainerStarted","Data":"0c4291ceada0819f0c4c9a5f6b946f2a8962fae34a0fc22a50f4a799f8a241b6"} Jul 20 19:28:35.806053 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:35.806007 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rcc5g" podStartSLOduration=67.156440016 podStartE2EDuration="1m11.805993266s" totalImagesPullingTime="4.64955325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:35.805322114 +0000 UTC m=+71.788615063" watchObservedRunningTime="2026-07-20 19:28:35.805993266 +0000 UTC m=+71.789286220" Jul 20 19:28:35.820405 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:28:35.820361 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pw8fv" podStartSLOduration=67.153615833 podStartE2EDuration="1m11.820346411s" totalImagesPullingTime="4.666730578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:28:35.819997421 +0000 UTC m=+71.803290377" watchObservedRunningTime="2026-07-20 19:28:35.820346411 +0000 UTC m=+71.803639365" Jul 20 19:29:02.754862 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:29:02.754736 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:29:02.754862 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:29:02.754828 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:29:02.755354 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:02.754882 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:29:02.755354 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:02.754903 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:29:02.755354 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:02.754926 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:29:02.755354 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:02.754970 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:30:06.754954915 +0000 UTC m=+162.738247848 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:29:02.755354 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:02.754983 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:30:06.754977055 +0000 UTC m=+162.738269988 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:29:02.855715 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:29:02.855681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:29:02.855888 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:02.855815 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:29:02.855888 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:02.855877 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:30:06.8558612 +0000 UTC m=+162.839154139 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:29:06.797302 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:29:06.797274 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rcc5g" Jul 20 19:29:34.379328 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:29:34.379291 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:29:34.379907 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:34.379469 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 19:29:34.379907 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:29:34.379554 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:31:36.379532914 +0000 UTC m=+252.362825853 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : secret "metrics-daemon-secret" not found Jul 20 19:30:01.919715 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:01.919662 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" podUID="cb64f499-8e30-48b7-8bbd-7312d893df9e" Jul 20 19:30:01.927837 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:01.927804 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" Jul 20 19:30:01.936958 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:01.936930 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6gzpv" podUID="d45325d1-bac5-4328-abc6-3faa098afeb1" Jul 20 19:30:01.988894 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:30:01.988871 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:30:01.989017 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:30:01.988871 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:30:01.989063 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:30:01.988872 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gzpv" Jul 20 19:30:03.542232 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:03.542190 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:30:06.811837 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:30:06.811798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:30:06.812237 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:30:06.811852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:30:06.812237 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:06.811941 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 19:30:06.812237 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:06.811947 2573 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 20 19:30:06.812237 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:06.812029 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert podName:cb64f499-8e30-48b7-8bbd-7312d893df9e nodeName:}" failed. No retries permitted until 2026-07-20 19:32:08.812012873 +0000 UTC m=+284.795305807 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-8vvwl" (UID: "cb64f499-8e30-48b7-8bbd-7312d893df9e") : secret "networking-console-plugin-cert" not found Jul 20 19:30:06.812237 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:06.811953 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b54cbdc8b-htc59: secret "image-registry-tls" not found Jul 20 19:30:06.812237 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:06.812092 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls podName:b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12 nodeName:}" failed. No retries permitted until 2026-07-20 19:32:08.812078028 +0000 UTC m=+284.795370962 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls") pod "image-registry-6b54cbdc8b-htc59" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12") : secret "image-registry-tls" not found Jul 20 19:30:06.912973 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:30:06.912941 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:30:06.913169 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:06.913111 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 19:30:06.913235 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:30:06.913190 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls podName:d45325d1-bac5-4328-abc6-3faa098afeb1 nodeName:}" failed. No retries permitted until 2026-07-20 19:32:08.913166953 +0000 UTC m=+284.896459897 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls") pod "dns-default-6gzpv" (UID: "d45325d1-bac5-4328-abc6-3faa098afeb1") : secret "dns-default-metrics-tls" not found Jul 20 19:30:15.530818 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:30:15.530769 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:31:36.418255 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:36.418216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:31:36.418752 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:36.418372 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 20 19:31:36.418752 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:36.418445 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs podName:d23a8cc4-8443-4595-9cf4-5dcacf92cc3f nodeName:}" failed. No retries permitted until 2026-07-20 19:33:38.418426603 +0000 UTC m=+374.401719536 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs") pod "network-metrics-daemon-qndtv" (UID: "d23a8cc4-8443-4595-9cf4-5dcacf92cc3f") : secret "metrics-daemon-secret" not found Jul 20 19:31:39.029739 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.029701 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv"] Jul 20 19:31:39.033161 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.033013 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.035200 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.034568 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm"] Jul 20 19:31:39.037376 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.037352 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm"] Jul 20 19:31:39.037498 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.037441 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:39.038340 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.038320 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 20 19:31:39.038472 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.038388 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-sfnvl\"" Jul 20 19:31:39.038472 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.038436 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:31:39.038702 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.038478 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 20 19:31:39.038746 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.038713 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 20 19:31:39.039105 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.039087 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pn76z"] Jul 20 19:31:39.039207 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.039191 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.039268 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.039248 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 20 19:31:39.040910 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.040891 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.041868 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.041834 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 20 19:31:39.042556 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.042536 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 20 19:31:39.042875 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.042861 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ghbhl\"" Jul 20 19:31:39.043282 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.043269 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:31:39.043467 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.043455 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tppcx\"" Jul 20 19:31:39.047262 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.047240 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 20 19:31:39.047371 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.047278 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 20 19:31:39.047371 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.047244 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 20 19:31:39.047371 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.047252 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:31:39.047523 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.047377 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6h4ws\"" Jul 20 19:31:39.047591 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.047577 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 19:31:39.047646 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.047627 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 19:31:39.048291 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.048274 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 20 19:31:39.051686 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.051665 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 20 19:31:39.055088 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.055066 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv"] Jul 20 19:31:39.057275 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.057253 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pn76z"] Jul 20 19:31:39.058033 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.058013 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm"] Jul 20 19:31:39.070974 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.070953 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm"] Jul 20 19:31:39.135337 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lffpj\" (UniqueName: \"kubernetes.io/projected/ba2123c0-fdb3-4b84-89fb-94a9467aa041-kube-api-access-lffpj\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.135518 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.135518 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135377 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wplgt\" (UniqueName: \"kubernetes.io/projected/857f9a2d-27aa-4663-92bb-64406b0da2a8-kube-api-access-wplgt\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.135518 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135420 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2123c0-fdb3-4b84-89fb-94a9467aa041-serving-cert\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.135518 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135460 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/857f9a2d-27aa-4663-92bb-64406b0da2a8-config\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.135518 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135488 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.135518 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135511 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxqxf\" (UniqueName: \"kubernetes.io/projected/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-kube-api-access-bxqxf\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.135779 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135567 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/857f9a2d-27aa-4663-92bb-64406b0da2a8-serving-cert\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.135779 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135629 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:39.135779 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9msg\" (UniqueName: \"kubernetes.io/projected/bca00f77-ea88-401a-b7d3-3e8acca7a28d-kube-api-access-h9msg\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:39.135779 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135673 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/857f9a2d-27aa-4663-92bb-64406b0da2a8-trusted-ca\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.135779 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.135694 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ba2123c0-fdb3-4b84-89fb-94a9467aa041-config\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.174990 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.174952 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb"] Jul 20 19:31:39.177164 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.177144 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" Jul 20 19:31:39.182211 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.182185 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"volume-data-source-validator-dockercfg-2nc9h\" is forbidden: User \"system:node:ip-10-0-128-93.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-93.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2nc9h\"" type="*v1.Secret" Jul 20 19:31:39.182490 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.182468 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-93.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-93.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 20 19:31:39.182565 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.182499 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"volume-data-source-validator-78cf4c6f6-vnzxb\" is forbidden: User \"system:node:ip-10-0-128-93.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-93.ec2.internal' and this object" podUID="13f07043-3b16-4458-89cc-a407ca185161" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" Jul 20 19:31:39.183278 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.183250 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-93.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-cluster-storage-operator\": no relationship found between node 'ip-10-0-128-93.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 20 19:31:39.204283 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.204255 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb"] Jul 20 19:31:39.236524 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236488 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lffpj\" (UniqueName: \"kubernetes.io/projected/ba2123c0-fdb3-4b84-89fb-94a9467aa041-kube-api-access-lffpj\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.236730 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236543 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.236730 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236568 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wplgt\" (UniqueName: \"kubernetes.io/projected/857f9a2d-27aa-4663-92bb-64406b0da2a8-kube-api-access-wplgt\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.236730 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236593 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2123c0-fdb3-4b84-89fb-94a9467aa041-serving-cert\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.236886 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.236763 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:39.236886 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/857f9a2d-27aa-4663-92bb-64406b0da2a8-config\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.236886 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.236831 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls podName:3f02c2e1-b01b-441b-9254-6e4a9a286d9c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:39.736815993 +0000 UTC m=+255.720108929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pbfvm" (UID: "3f02c2e1-b01b-441b-9254-6e4a9a286d9c") : secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:39.236886 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236870 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236890 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxqxf\" (UniqueName: \"kubernetes.io/projected/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-kube-api-access-bxqxf\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236912 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/857f9a2d-27aa-4663-92bb-64406b0da2a8-serving-cert\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.236968 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9msg\" (UniqueName: \"kubernetes.io/projected/bca00f77-ea88-401a-b7d3-3e8acca7a28d-kube-api-access-h9msg\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.237001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/857f9a2d-27aa-4663-92bb-64406b0da2a8-trusted-ca\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.237030 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ba2123c0-fdb3-4b84-89fb-94a9467aa041-config\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.237048 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 19:31:39.237103 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.237109 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls podName:bca00f77-ea88-401a-b7d3-3e8acca7a28d nodeName:}" failed. No retries permitted until 2026-07-20 19:31:39.737091869 +0000 UTC m=+255.720384814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jvfkm" (UID: "bca00f77-ea88-401a-b7d3-3e8acca7a28d") : secret "samples-operator-tls" not found Jul 20 19:31:39.237669 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.237641 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/857f9a2d-27aa-4663-92bb-64406b0da2a8-config\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.237777 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.237681 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ba2123c0-fdb3-4b84-89fb-94a9467aa041-config\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.237777 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.237751 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.237871 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.237830 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/857f9a2d-27aa-4663-92bb-64406b0da2a8-trusted-ca\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.239008 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.238993 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ba2123c0-fdb3-4b84-89fb-94a9467aa041-serving-cert\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.239480 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.239458 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/857f9a2d-27aa-4663-92bb-64406b0da2a8-serving-cert\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.245577 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.245549 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lffpj\" (UniqueName: \"kubernetes.io/projected/ba2123c0-fdb3-4b84-89fb-94a9467aa041-kube-api-access-lffpj\") pod \"service-ca-operator-5bcbc94c78-c5zqv\" (UID: \"ba2123c0-fdb3-4b84-89fb-94a9467aa041\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.245878 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.245855 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wplgt\" (UniqueName: \"kubernetes.io/projected/857f9a2d-27aa-4663-92bb-64406b0da2a8-kube-api-access-wplgt\") pod \"console-operator-565b7b569d-pn76z\" (UID: \"857f9a2d-27aa-4663-92bb-64406b0da2a8\") " pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.245963 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.245925 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxqxf\" (UniqueName: \"kubernetes.io/projected/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-kube-api-access-bxqxf\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.246086 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.246067 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9msg\" (UniqueName: \"kubernetes.io/projected/bca00f77-ea88-401a-b7d3-3e8acca7a28d-kube-api-access-h9msg\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:39.338216 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.338118 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b6tw\" (UniqueName: \"kubernetes.io/projected/13f07043-3b16-4458-89cc-a407ca185161-kube-api-access-6b6tw\") pod \"volume-data-source-validator-78cf4c6f6-vnzxb\" (UID: \"13f07043-3b16-4458-89cc-a407ca185161\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" Jul 20 19:31:39.345074 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.345053 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" Jul 20 19:31:39.364084 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.364056 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:39.439367 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.439332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b6tw\" (UniqueName: \"kubernetes.io/projected/13f07043-3b16-4458-89cc-a407ca185161-kube-api-access-6b6tw\") pod \"volume-data-source-validator-78cf4c6f6-vnzxb\" (UID: \"13f07043-3b16-4458-89cc-a407ca185161\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" Jul 20 19:31:39.472906 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.472826 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv"] Jul 20 19:31:39.477033 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:31:39.477010 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2123c0_fdb3_4b84_89fb_94a9467aa041.slice/crio-f134841472f9da620d72d89758fe197db2ae7fd5b8384b7435cd5ce56e17e5de WatchSource:0}: Error finding container f134841472f9da620d72d89758fe197db2ae7fd5b8384b7435cd5ce56e17e5de: Status 404 returned error can't find the container with id f134841472f9da620d72d89758fe197db2ae7fd5b8384b7435cd5ce56e17e5de Jul 20 19:31:39.491133 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.491101 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-pn76z"] Jul 20 19:31:39.493428 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:31:39.493403 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857f9a2d_27aa_4663_92bb_64406b0da2a8.slice/crio-5bcb44ff4d09df6f94f238ac9f80d7bf6bd28d1e1ded3a0e7ba2032d0283f669 WatchSource:0}: Error finding container 5bcb44ff4d09df6f94f238ac9f80d7bf6bd28d1e1ded3a0e7ba2032d0283f669: Status 404 returned error can't find the container with id 5bcb44ff4d09df6f94f238ac9f80d7bf6bd28d1e1ded3a0e7ba2032d0283f669 Jul 20 19:31:39.741398 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.741349 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:39.741577 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:39.741439 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:39.741577 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.741497 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 19:31:39.741577 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.741539 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:39.741577 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.741557 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls podName:bca00f77-ea88-401a-b7d3-3e8acca7a28d nodeName:}" failed. No retries permitted until 2026-07-20 19:31:40.741542529 +0000 UTC m=+256.724835462 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jvfkm" (UID: "bca00f77-ea88-401a-b7d3-3e8acca7a28d") : secret "samples-operator-tls" not found Jul 20 19:31:39.741577 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:39.741579 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls podName:3f02c2e1-b01b-441b-9254-6e4a9a286d9c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:40.741567826 +0000 UTC m=+256.724860760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pbfvm" (UID: "3f02c2e1-b01b-441b-9254-6e4a9a286d9c") : secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:40.155719 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.155686 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2nc9h\"" Jul 20 19:31:40.214841 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.214780 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" event={"ID":"ba2123c0-fdb3-4b84-89fb-94a9467aa041","Type":"ContainerStarted","Data":"f134841472f9da620d72d89758fe197db2ae7fd5b8384b7435cd5ce56e17e5de"} Jul 20 19:31:40.216229 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.216200 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" event={"ID":"857f9a2d-27aa-4663-92bb-64406b0da2a8","Type":"ContainerStarted","Data":"5bcb44ff4d09df6f94f238ac9f80d7bf6bd28d1e1ded3a0e7ba2032d0283f669"} Jul 20 19:31:40.329871 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.329832 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 20 19:31:40.418868 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.418773 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:31:40.429492 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.429463 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b6tw\" (UniqueName: \"kubernetes.io/projected/13f07043-3b16-4458-89cc-a407ca185161-kube-api-access-6b6tw\") pod \"volume-data-source-validator-78cf4c6f6-vnzxb\" (UID: \"13f07043-3b16-4458-89cc-a407ca185161\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" Jul 20 19:31:40.685396 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.685314 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" Jul 20 19:31:40.754488 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.753780 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:40.754488 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.753877 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:40.754488 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:40.754034 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 19:31:40.754488 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:40.754055 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:40.754488 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:40.754105 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls podName:bca00f77-ea88-401a-b7d3-3e8acca7a28d nodeName:}" failed. No retries permitted until 2026-07-20 19:31:42.754082828 +0000 UTC m=+258.737375785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jvfkm" (UID: "bca00f77-ea88-401a-b7d3-3e8acca7a28d") : secret "samples-operator-tls" not found Jul 20 19:31:40.754488 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:40.754143 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls podName:3f02c2e1-b01b-441b-9254-6e4a9a286d9c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:42.754117218 +0000 UTC m=+258.737410170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pbfvm" (UID: "3f02c2e1-b01b-441b-9254-6e4a9a286d9c") : secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:40.828394 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:40.828356 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb"] Jul 20 19:31:40.831930 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:31:40.831900 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13f07043_3b16_4458_89cc_a407ca185161.slice/crio-1b69bf85872e58b42f59f8da0a0b3a9392cbba337d96771d420f891e7f018a7b WatchSource:0}: Error finding container 1b69bf85872e58b42f59f8da0a0b3a9392cbba337d96771d420f891e7f018a7b: Status 404 returned error can't find the container with id 1b69bf85872e58b42f59f8da0a0b3a9392cbba337d96771d420f891e7f018a7b Jul 20 19:31:41.220583 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:41.220541 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" event={"ID":"13f07043-3b16-4458-89cc-a407ca185161","Type":"ContainerStarted","Data":"1b69bf85872e58b42f59f8da0a0b3a9392cbba337d96771d420f891e7f018a7b"} Jul 20 19:31:42.224565 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:42.224536 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" event={"ID":"ba2123c0-fdb3-4b84-89fb-94a9467aa041","Type":"ContainerStarted","Data":"ac5997e63204fb6a59d8ef85c8a0a402324425dfab72108fec14600a682df077"} Jul 20 19:31:42.772169 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:42.772130 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:42.772356 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:42.772241 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:42.772356 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:42.772301 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:42.772356 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:42.772340 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 19:31:42.772525 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:42.772382 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls podName:3f02c2e1-b01b-441b-9254-6e4a9a286d9c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:46.772360764 +0000 UTC m=+262.755653710 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pbfvm" (UID: "3f02c2e1-b01b-441b-9254-6e4a9a286d9c") : secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:42.772525 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:42.772417 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls podName:bca00f77-ea88-401a-b7d3-3e8acca7a28d nodeName:}" failed. No retries permitted until 2026-07-20 19:31:46.772395522 +0000 UTC m=+262.755688459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jvfkm" (UID: "bca00f77-ea88-401a-b7d3-3e8acca7a28d") : secret "samples-operator-tls" not found Jul 20 19:31:43.230721 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.230683 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" event={"ID":"13f07043-3b16-4458-89cc-a407ca185161","Type":"ContainerStarted","Data":"4d23cab2f1d9a69c50e478a427d3dd28e698827189ad28f88a6f66de6c93d440"} Jul 20 19:31:43.232187 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.232168 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/0.log" Jul 20 19:31:43.232290 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.232204 2573 generic.go:356] "Generic (PLEG): container finished" podID="857f9a2d-27aa-4663-92bb-64406b0da2a8" containerID="9cd832e141324fdd67699524dff7b235a5590288aa5de95b167ae60e31972d56" exitCode=255 Jul 20 19:31:43.232290 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.232271 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" event={"ID":"857f9a2d-27aa-4663-92bb-64406b0da2a8","Type":"ContainerDied","Data":"9cd832e141324fdd67699524dff7b235a5590288aa5de95b167ae60e31972d56"} Jul 20 19:31:43.232479 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.232466 2573 scope.go:117] "RemoveContainer" containerID="9cd832e141324fdd67699524dff7b235a5590288aa5de95b167ae60e31972d56" Jul 20 19:31:43.246750 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.246703 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vnzxb" podStartSLOduration=2.186828335 podStartE2EDuration="4.246683761s" totalImagesPullingTime="2.059855426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:43.245586559 +0000 UTC m=+259.228879538" watchObservedRunningTime="2026-07-20 19:31:43.246683761 +0000 UTC m=+259.229976717" Jul 20 19:31:43.262850 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.262811 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" podStartSLOduration=1.575597366 podStartE2EDuration="4.262798355s" totalImagesPullingTime="2.687200989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:43.262142376 +0000 UTC m=+259.245435332" watchObservedRunningTime="2026-07-20 19:31:43.262798355 +0000 UTC m=+259.246091310" Jul 20 19:31:43.407585 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.407553 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-czg79"] Jul 20 19:31:43.409553 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.409538 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.412292 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.412263 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 19:31:43.412292 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.412244 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8lkqn\"" Jul 20 19:31:43.412489 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.412332 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 19:31:43.412573 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.412522 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 19:31:43.412728 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.412688 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 19:31:43.429795 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.429769 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-czg79"] Jul 20 19:31:43.580275 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.580169 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.580275 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.580227 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-splq5\" (UniqueName: \"kubernetes.io/projected/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-api-access-splq5\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.580275 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.580251 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-data-volume\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.580275 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.580286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-crio-socket\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.580600 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.580332 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.664675 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.664642 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb"] Jul 20 19:31:43.666743 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.666728 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" Jul 20 19:31:43.668811 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.668788 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 20 19:31:43.668960 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.668941 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-9spdz\"" Jul 20 19:31:43.669023 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.668963 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 20 19:31:43.676566 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.676540 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb"] Jul 20 19:31:43.680867 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.680847 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-splq5\" (UniqueName: \"kubernetes.io/projected/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-api-access-splq5\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.680990 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.680875 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-data-volume\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.680990 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.680896 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-crio-socket\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.680990 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.680981 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-crio-socket\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.681135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.680981 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.681135 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:43.681033 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:43.681135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.681086 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.681135 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:43.681099 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:44.181079845 +0000 UTC m=+260.164372782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : configmap "kube-rbac-proxy" not found Jul 20 19:31:43.681360 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:43.681160 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:43.681360 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:43.681206 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:44.181192296 +0000 UTC m=+260.164485234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:43.681360 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.681204 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-data-volume\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.697429 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.697403 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-splq5\" (UniqueName: \"kubernetes.io/projected/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-api-access-splq5\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:43.781874 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.781837 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s6ct\" (UniqueName: \"kubernetes.io/projected/148bacec-cc15-4a1a-9e7c-a3ff522ce323-kube-api-access-8s6ct\") pod \"migrator-7cc9c7ffd7-6dmpb\" (UID: \"148bacec-cc15-4a1a-9e7c-a3ff522ce323\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" Jul 20 19:31:43.882979 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.882942 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s6ct\" (UniqueName: \"kubernetes.io/projected/148bacec-cc15-4a1a-9e7c-a3ff522ce323-kube-api-access-8s6ct\") pod \"migrator-7cc9c7ffd7-6dmpb\" (UID: \"148bacec-cc15-4a1a-9e7c-a3ff522ce323\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" Jul 20 19:31:43.891183 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.891151 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s6ct\" (UniqueName: \"kubernetes.io/projected/148bacec-cc15-4a1a-9e7c-a3ff522ce323-kube-api-access-8s6ct\") pod \"migrator-7cc9c7ffd7-6dmpb\" (UID: \"148bacec-cc15-4a1a-9e7c-a3ff522ce323\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" Jul 20 19:31:43.975310 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:43.975278 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" Jul 20 19:31:44.100777 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.100739 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb"] Jul 20 19:31:44.103866 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:31:44.103840 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148bacec_cc15_4a1a_9e7c_a3ff522ce323.slice/crio-707b17458820de50c6608bde2d4546deb72c1d6ff9ae74dbd658b12c094c6ce1 WatchSource:0}: Error finding container 707b17458820de50c6608bde2d4546deb72c1d6ff9ae74dbd658b12c094c6ce1: Status 404 returned error can't find the container with id 707b17458820de50c6608bde2d4546deb72c1d6ff9ae74dbd658b12c094c6ce1 Jul 20 19:31:44.185682 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.185594 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:44.185793 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.185694 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:44.185793 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:44.185761 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:44.185793 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:44.185784 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:44.185896 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:44.185830 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:45.185811374 +0000 UTC m=+261.169104314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:44.185896 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:44.185845 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:45.185838843 +0000 UTC m=+261.169131779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : configmap "kube-rbac-proxy" not found Jul 20 19:31:44.237380 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.237353 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/1.log" Jul 20 19:31:44.237764 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.237732 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/0.log" Jul 20 19:31:44.237801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.237761 2573 generic.go:356] "Generic (PLEG): container finished" podID="857f9a2d-27aa-4663-92bb-64406b0da2a8" containerID="3c941796dd4c3b0650fb7639d715d2171061d6f49373a8787863786a4757dee4" exitCode=255 Jul 20 19:31:44.237849 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.237835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" event={"ID":"857f9a2d-27aa-4663-92bb-64406b0da2a8","Type":"ContainerDied","Data":"3c941796dd4c3b0650fb7639d715d2171061d6f49373a8787863786a4757dee4"} Jul 20 19:31:44.237901 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.237889 2573 scope.go:117] "RemoveContainer" containerID="9cd832e141324fdd67699524dff7b235a5590288aa5de95b167ae60e31972d56" Jul 20 19:31:44.238120 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.238094 2573 scope.go:117] "RemoveContainer" containerID="3c941796dd4c3b0650fb7639d715d2171061d6f49373a8787863786a4757dee4" Jul 20 19:31:44.238288 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:44.238264 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pn76z_openshift-console-operator(857f9a2d-27aa-4663-92bb-64406b0da2a8)\"" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" podUID="857f9a2d-27aa-4663-92bb-64406b0da2a8" Jul 20 19:31:44.239212 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:44.239175 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" event={"ID":"148bacec-cc15-4a1a-9e7c-a3ff522ce323","Type":"ContainerStarted","Data":"707b17458820de50c6608bde2d4546deb72c1d6ff9ae74dbd658b12c094c6ce1"} Jul 20 19:31:45.191250 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:45.191215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:45.191406 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:45.191288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:45.191406 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:45.191344 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:45.191471 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:45.191415 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:47.191400549 +0000 UTC m=+263.174693481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : configmap "kube-rbac-proxy" not found Jul 20 19:31:45.191471 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:45.191421 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:45.191471 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:45.191462 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:47.191448268 +0000 UTC m=+263.174741201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:45.242785 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:45.242762 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/1.log" Jul 20 19:31:45.243057 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:45.243042 2573 scope.go:117] "RemoveContainer" containerID="3c941796dd4c3b0650fb7639d715d2171061d6f49373a8787863786a4757dee4" Jul 20 19:31:45.243203 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:45.243185 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pn76z_openshift-console-operator(857f9a2d-27aa-4663-92bb-64406b0da2a8)\"" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" podUID="857f9a2d-27aa-4663-92bb-64406b0da2a8" Jul 20 19:31:45.594825 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:45.594737 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qxwxz_0d00559d-ac72-41c0-ab2d-68879013c63a/dns-node-resolver/0.log" Jul 20 19:31:46.247559 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:46.247481 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" event={"ID":"148bacec-cc15-4a1a-9e7c-a3ff522ce323","Type":"ContainerStarted","Data":"f58c5d444639614187a57676bc8c08ac2d59a6aa4823f3c9fd1f5880b8142cc2"} Jul 20 19:31:46.247559 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:46.247534 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" event={"ID":"148bacec-cc15-4a1a-9e7c-a3ff522ce323","Type":"ContainerStarted","Data":"4ba8d65ed4c9f4d4f952996771d6899df85a78738644eedfc1bf6d005db08c49"} Jul 20 19:31:46.263111 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:46.263064 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-6dmpb" podStartSLOduration=2.166348933 podStartE2EDuration="3.263050255s" totalImagesPullingTime="1.096701322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:46.262637499 +0000 UTC m=+262.245930477" watchObservedRunningTime="2026-07-20 19:31:46.263050255 +0000 UTC m=+262.246343209" Jul 20 19:31:46.394447 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:46.394419 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ng274_d7dc7bed-e5e1-494e-af3e-3420e9712c1b/node-ca/0.log" Jul 20 19:31:46.806076 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:46.806046 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:46.806230 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:46.806111 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:46.806230 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:46.806190 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:46.806339 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:46.806251 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls podName:3f02c2e1-b01b-441b-9254-6e4a9a286d9c nodeName:}" failed. No retries permitted until 2026-07-20 19:31:54.806236012 +0000 UTC m=+270.789528945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pbfvm" (UID: "3f02c2e1-b01b-441b-9254-6e4a9a286d9c") : secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:46.806339 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:46.806194 2573 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 19:31:46.806339 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:46.806316 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls podName:bca00f77-ea88-401a-b7d3-3e8acca7a28d nodeName:}" failed. No retries permitted until 2026-07-20 19:31:54.806304993 +0000 UTC m=+270.789597926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-jvfkm" (UID: "bca00f77-ea88-401a-b7d3-3e8acca7a28d") : secret "samples-operator-tls" not found Jul 20 19:31:47.209714 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:47.209672 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:47.209930 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:47.209748 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:47.209930 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:47.209814 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:47.209930 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:47.209836 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:47.209930 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:47.209890 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:51.209869473 +0000 UTC m=+267.193162421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : configmap "kube-rbac-proxy" not found Jul 20 19:31:47.209930 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:47.209909 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:51.209901231 +0000 UTC m=+267.193194165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:47.395818 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:47.395789 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x2c5n_704c4f1f-46d5-4fc5-80ed-127d316b940b/serve-healthcheck-canary/0.log" Jul 20 19:31:49.364476 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:49.364449 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:49.364872 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:49.364499 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:31:49.364872 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:49.364842 2573 scope.go:117] "RemoveContainer" containerID="3c941796dd4c3b0650fb7639d715d2171061d6f49373a8787863786a4757dee4" Jul 20 19:31:49.364993 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:49.364975 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-pn76z_openshift-console-operator(857f9a2d-27aa-4663-92bb-64406b0da2a8)\"" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" podUID="857f9a2d-27aa-4663-92bb-64406b0da2a8" Jul 20 19:31:51.239806 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:51.239774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:51.240199 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:51.239857 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:51.240199 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:51.239916 2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 19:31:51.240199 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:51.239938 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:51.240199 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:51.239967 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:59.239954077 +0000 UTC m=+275.223247009 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : configmap "kube-rbac-proxy" not found Jul 20 19:31:51.240199 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:51.239989 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:31:59.239975798 +0000 UTC m=+275.223268732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:54.871603 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:54.871571 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:54.872022 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:54.871665 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:31:54.872022 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:54.871743 2573 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:54.872022 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:54.871795 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls podName:3f02c2e1-b01b-441b-9254-6e4a9a286d9c nodeName:}" failed. No retries permitted until 2026-07-20 19:32:10.871780423 +0000 UTC m=+286.855073359 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-pbfvm" (UID: "3f02c2e1-b01b-441b-9254-6e4a9a286d9c") : secret "cluster-monitoring-operator-tls" not found Jul 20 19:31:54.873931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:54.873910 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/bca00f77-ea88-401a-b7d3-3e8acca7a28d-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-jvfkm\" (UID: \"bca00f77-ea88-401a-b7d3-3e8acca7a28d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:54.952279 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:54.952248 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" Jul 20 19:31:55.074741 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:55.074712 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm"] Jul 20 19:31:55.275496 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:55.275412 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" event={"ID":"bca00f77-ea88-401a-b7d3-3e8acca7a28d","Type":"ContainerStarted","Data":"60a98c2acb12eb0d5852bb00f22e08b2397495bef096e3879751b02ba011f9f8"} Jul 20 19:31:57.283555 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:57.283520 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" event={"ID":"bca00f77-ea88-401a-b7d3-3e8acca7a28d","Type":"ContainerStarted","Data":"6d6135501535810d2b2f21f279996c55350562d0e7b81de0dda9915bb620f53a"} Jul 20 19:31:57.283974 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:57.283607 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" event={"ID":"bca00f77-ea88-401a-b7d3-3e8acca7a28d","Type":"ContainerStarted","Data":"031c101bb6412c080a089ab56834b8941ff8e852cfbdbfb4807caad02168b2d9"} Jul 20 19:31:57.300024 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:57.299942 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-jvfkm" podStartSLOduration=16.372844561 podStartE2EDuration="18.299927803s" totalImagesPullingTime="1.927083242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:31:57.298843346 +0000 UTC m=+273.282136301" watchObservedRunningTime="2026-07-20 19:31:57.299927803 +0000 UTC m=+273.283220757" Jul 20 19:31:59.307820 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:59.307791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:59.308273 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:59.307878 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:31:59.308273 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:59.308004 2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 19:31:59.308273 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:31:59.308073 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls podName:6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615 nodeName:}" failed. No retries permitted until 2026-07-20 19:32:15.308052397 +0000 UTC m=+291.291345354 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls") pod "insights-runtime-extractor-czg79" (UID: "6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615") : secret "insights-runtime-extractor-tls" not found Jul 20 19:31:59.308449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:31:59.308277 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:32:03.531596 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:03.531561 2573 scope.go:117] "RemoveContainer" containerID="3c941796dd4c3b0650fb7639d715d2171061d6f49373a8787863786a4757dee4" Jul 20 19:32:04.306717 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:04.306691 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:32:04.307111 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:04.307094 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/1.log" Jul 20 19:32:04.307186 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:04.307127 2573 generic.go:356] "Generic (PLEG): container finished" podID="857f9a2d-27aa-4663-92bb-64406b0da2a8" containerID="842e4992416f274bea1eacf72b1fa31ee4e58b46e33d571f1fcc7e2a6a5fb543" exitCode=255 Jul 20 19:32:04.307221 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:04.307185 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" event={"ID":"857f9a2d-27aa-4663-92bb-64406b0da2a8","Type":"ContainerDied","Data":"842e4992416f274bea1eacf72b1fa31ee4e58b46e33d571f1fcc7e2a6a5fb543"} Jul 20 19:32:04.307253 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:04.307228 2573 scope.go:117] "RemoveContainer" containerID="3c941796dd4c3b0650fb7639d715d2171061d6f49373a8787863786a4757dee4" Jul 20 19:32:04.307552 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:04.307532 2573 scope.go:117] "RemoveContainer" containerID="842e4992416f274bea1eacf72b1fa31ee4e58b46e33d571f1fcc7e2a6a5fb543" Jul 20 19:32:04.307714 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:04.307696 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-pn76z_openshift-console-operator(857f9a2d-27aa-4663-92bb-64406b0da2a8)\"" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" podUID="857f9a2d-27aa-4663-92bb-64406b0da2a8" Jul 20 19:32:04.989827 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:04.989789 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-6gzpv" podUID="d45325d1-bac5-4328-abc6-3faa098afeb1" Jul 20 19:32:04.989827 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:04.989808 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" podUID="cb64f499-8e30-48b7-8bbd-7312d893df9e" Jul 20 19:32:04.990230 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:04.989808 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" Jul 20 19:32:05.311360 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:05.311281 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:32:05.311510 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:05.311388 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gzpv" Jul 20 19:32:05.311510 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:05.311491 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:32:05.311581 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:05.311514 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:08.879899 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.879862 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:32:08.880285 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.879934 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:08.882520 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.882483 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"image-registry-6b54cbdc8b-htc59\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:08.882664 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.882486 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cb64f499-8e30-48b7-8bbd-7312d893df9e-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-8vvwl\" (UID: \"cb64f499-8e30-48b7-8bbd-7312d893df9e\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:32:08.915081 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.915058 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dpg8d\"" Jul 20 19:32:08.915170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.915058 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dn7g9\"" Jul 20 19:32:08.922920 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.922903 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" Jul 20 19:32:08.922974 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.922936 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:08.981819 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.981317 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:32:08.984447 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:08.984415 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d45325d1-bac5-4328-abc6-3faa098afeb1-metrics-tls\") pod \"dns-default-6gzpv\" (UID: \"d45325d1-bac5-4328-abc6-3faa098afeb1\") " pod="openshift-dns/dns-default-6gzpv" Jul 20 19:32:09.058853 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.058716 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl"] Jul 20 19:32:09.061318 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:09.061292 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb64f499_8e30_48b7_8bbd_7312d893df9e.slice/crio-e43c547c68c8c7065fd78da00ae16acacfb65438b33cdbe21f28b17c7c9ad754 WatchSource:0}: Error finding container e43c547c68c8c7065fd78da00ae16acacfb65438b33cdbe21f28b17c7c9ad754: Status 404 returned error can't find the container with id e43c547c68c8c7065fd78da00ae16acacfb65438b33cdbe21f28b17c7c9ad754 Jul 20 19:32:09.078355 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.078334 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b54cbdc8b-htc59"] Jul 20 19:32:09.081141 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:09.081118 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3ffb3ec_d116_48ee_9c7f_f1d8b294ca12.slice/crio-1b6cfd1d9844fff0c1c75ec7e7280567321ed126da3e9a9e5f9ee60cc55d7f16 WatchSource:0}: Error finding container 1b6cfd1d9844fff0c1c75ec7e7280567321ed126da3e9a9e5f9ee60cc55d7f16: Status 404 returned error can't find the container with id 1b6cfd1d9844fff0c1c75ec7e7280567321ed126da3e9a9e5f9ee60cc55d7f16 Jul 20 19:32:09.214998 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.214919 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-27sg8\"" Jul 20 19:32:09.223043 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.223018 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6gzpv" Jul 20 19:32:09.331973 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.331934 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" event={"ID":"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12","Type":"ContainerStarted","Data":"59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda"} Jul 20 19:32:09.332134 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.332001 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" event={"ID":"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12","Type":"ContainerStarted","Data":"1b6cfd1d9844fff0c1c75ec7e7280567321ed126da3e9a9e5f9ee60cc55d7f16"} Jul 20 19:32:09.332134 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.332063 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:09.333716 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.333679 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" event={"ID":"cb64f499-8e30-48b7-8bbd-7312d893df9e","Type":"ContainerStarted","Data":"e43c547c68c8c7065fd78da00ae16acacfb65438b33cdbe21f28b17c7c9ad754"} Jul 20 19:32:09.357207 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.356105 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" podStartSLOduration=263.356089092 podStartE2EDuration="4m23.356089092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:09.355326344 +0000 UTC m=+285.338619299" watchObservedRunningTime="2026-07-20 19:32:09.356089092 +0000 UTC m=+285.339382048" Jul 20 19:32:09.358522 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.358496 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6gzpv"] Jul 20 19:32:09.359870 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:09.359847 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45325d1_bac5_4328_abc6_3faa098afeb1.slice/crio-70573a0ce7726a7ad31d913d79087a398decd381d8c6b1b633b7b8ecb68bd30b WatchSource:0}: Error finding container 70573a0ce7726a7ad31d913d79087a398decd381d8c6b1b633b7b8ecb68bd30b: Status 404 returned error can't find the container with id 70573a0ce7726a7ad31d913d79087a398decd381d8c6b1b633b7b8ecb68bd30b Jul 20 19:32:09.364148 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.364132 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:32:09.364229 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.364169 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:32:09.364431 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:09.364416 2573 scope.go:117] "RemoveContainer" containerID="842e4992416f274bea1eacf72b1fa31ee4e58b46e33d571f1fcc7e2a6a5fb543" Jul 20 19:32:09.364565 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:09.364548 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-pn76z_openshift-console-operator(857f9a2d-27aa-4663-92bb-64406b0da2a8)\"" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" podUID="857f9a2d-27aa-4663-92bb-64406b0da2a8" Jul 20 19:32:10.339177 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:10.339133 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gzpv" event={"ID":"d45325d1-bac5-4328-abc6-3faa098afeb1","Type":"ContainerStarted","Data":"70573a0ce7726a7ad31d913d79087a398decd381d8c6b1b633b7b8ecb68bd30b"} Jul 20 19:32:10.899744 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:10.899704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:32:10.902131 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:10.902098 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3f02c2e1-b01b-441b-9254-6e4a9a286d9c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-pbfvm\" (UID: \"3f02c2e1-b01b-441b-9254-6e4a9a286d9c\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:32:11.159374 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:11.159298 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" Jul 20 19:32:11.343583 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:11.343551 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" event={"ID":"cb64f499-8e30-48b7-8bbd-7312d893df9e","Type":"ContainerStarted","Data":"82b8c6e6ea4c34d4c851a0cd7c727771afc265405a719daa863d9d7295781cc3"} Jul 20 19:32:11.362602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:11.362556 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-8vvwl" podStartSLOduration=271.061432393 podStartE2EDuration="4m32.362541704s" totalImagesPullingTime="1.301109311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:11.362459036 +0000 UTC m=+287.345751990" watchObservedRunningTime="2026-07-20 19:32:11.362541704 +0000 UTC m=+287.345834658" Jul 20 19:32:11.517580 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:11.517548 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm"] Jul 20 19:32:11.520506 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:11.520483 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f02c2e1_b01b_441b_9254_6e4a9a286d9c.slice/crio-8ef945383f37c4579cd99add87adba16514ea94db58b558ef5e588a1e99d96bd WatchSource:0}: Error finding container 8ef945383f37c4579cd99add87adba16514ea94db58b558ef5e588a1e99d96bd: Status 404 returned error can't find the container with id 8ef945383f37c4579cd99add87adba16514ea94db58b558ef5e588a1e99d96bd Jul 20 19:32:12.349593 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:12.349505 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gzpv" event={"ID":"d45325d1-bac5-4328-abc6-3faa098afeb1","Type":"ContainerStarted","Data":"bcb3b30ff41ae91913e270d8281589b68282a980c9ad0b345623202f1ba45c4b"} Jul 20 19:32:12.350049 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:12.349599 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6gzpv" event={"ID":"d45325d1-bac5-4328-abc6-3faa098afeb1","Type":"ContainerStarted","Data":"b16df15696a3dc8eee5680061b243e959e16119c381cc9f5f84a076d88a89d2d"} Jul 20 19:32:12.350049 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:12.349644 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6gzpv" Jul 20 19:32:12.350804 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:12.350779 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" event={"ID":"3f02c2e1-b01b-441b-9254-6e4a9a286d9c","Type":"ContainerStarted","Data":"8ef945383f37c4579cd99add87adba16514ea94db58b558ef5e588a1e99d96bd"} Jul 20 19:32:12.367188 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:12.367128 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6gzpv" podStartSLOduration=252.001559081 podStartE2EDuration="4m14.367110562s" totalImagesPullingTime="2.365551481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:12.366734232 +0000 UTC m=+288.350027187" watchObservedRunningTime="2026-07-20 19:32:12.367110562 +0000 UTC m=+288.350403518" Jul 20 19:32:13.356538 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.356488 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" event={"ID":"3f02c2e1-b01b-441b-9254-6e4a9a286d9c","Type":"ContainerStarted","Data":"6d4d5c84f4a685137ac1e80e9922010b94c1992a47835df29f1a220259079807"} Jul 20 19:32:13.374463 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.374400 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-pbfvm" podStartSLOduration=32.681216031 podStartE2EDuration="34.37437837s" totalImagesPullingTime="1.693162339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:13.373068297 +0000 UTC m=+289.356361267" watchObservedRunningTime="2026-07-20 19:32:13.37437837 +0000 UTC m=+289.357671326" Jul 20 19:32:13.709111 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.709078 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2"] Jul 20 19:32:13.711368 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.711353 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:13.713492 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.713471 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-gklt9\"" Jul 20 19:32:13.713756 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.713733 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 20 19:32:13.720703 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.720682 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2"] Jul 20 19:32:13.826092 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.826055 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c78be445-1716-4837-8071-338258d7afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-fbzc2\" (UID: \"c78be445-1716-4837-8071-338258d7afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:13.927276 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:13.927243 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c78be445-1716-4837-8071-338258d7afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-fbzc2\" (UID: \"c78be445-1716-4837-8071-338258d7afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:13.927414 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:13.927358 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 20 19:32:13.927452 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:13.927421 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78be445-1716-4837-8071-338258d7afcb-tls-certificates podName:c78be445-1716-4837-8071-338258d7afcb nodeName:}" failed. No retries permitted until 2026-07-20 19:32:14.427406759 +0000 UTC m=+290.410699696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c78be445-1716-4837-8071-338258d7afcb-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-fbzc2" (UID: "c78be445-1716-4837-8071-338258d7afcb") : secret "prometheus-operator-admission-webhook-tls" not found Jul 20 19:32:14.431518 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:14.431474 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c78be445-1716-4837-8071-338258d7afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-fbzc2\" (UID: \"c78be445-1716-4837-8071-338258d7afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:14.434060 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:14.434038 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c78be445-1716-4837-8071-338258d7afcb-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-fbzc2\" (UID: \"c78be445-1716-4837-8071-338258d7afcb\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:14.619922 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:14.619888 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:14.745047 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:14.745015 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2"] Jul 20 19:32:14.748579 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:14.748553 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78be445_1716_4837_8071_338258d7afcb.slice/crio-3c240b24fb6d321a92556fd22188ea8800e561e8809f0a60c95fc4301dd0f58c WatchSource:0}: Error finding container 3c240b24fb6d321a92556fd22188ea8800e561e8809f0a60c95fc4301dd0f58c: Status 404 returned error can't find the container with id 3c240b24fb6d321a92556fd22188ea8800e561e8809f0a60c95fc4301dd0f58c Jul 20 19:32:15.338441 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:15.338402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:32:15.340713 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:15.340693 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-czg79\" (UID: \"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615\") " pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:32:15.363855 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:15.363822 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" event={"ID":"c78be445-1716-4837-8071-338258d7afcb","Type":"ContainerStarted","Data":"3c240b24fb6d321a92556fd22188ea8800e561e8809f0a60c95fc4301dd0f58c"} Jul 20 19:32:15.536544 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:15.536504 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-czg79" Jul 20 19:32:15.672084 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:15.671849 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-czg79"] Jul 20 19:32:15.674292 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:15.674258 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2fe0d0_d05a_4a2d_b34d_31a3cea3e615.slice/crio-e29f0e156b169566fdaad11cffad7cf93aba688d1798027742a3c779164dc87c WatchSource:0}: Error finding container e29f0e156b169566fdaad11cffad7cf93aba688d1798027742a3c779164dc87c: Status 404 returned error can't find the container with id e29f0e156b169566fdaad11cffad7cf93aba688d1798027742a3c779164dc87c Jul 20 19:32:16.372042 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.371676 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" event={"ID":"c78be445-1716-4837-8071-338258d7afcb","Type":"ContainerStarted","Data":"e5d315b2ea5b61520a600f1e34579ebb50fc7534c4ee69c79342999f86e878a7"} Jul 20 19:32:16.374020 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.372457 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:16.377681 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.374324 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czg79" event={"ID":"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615","Type":"ContainerStarted","Data":"e29f0e156b169566fdaad11cffad7cf93aba688d1798027742a3c779164dc87c"} Jul 20 19:32:16.378916 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.378871 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" Jul 20 19:32:16.404715 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.404653 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-fbzc2" podStartSLOduration=2.09144363 podStartE2EDuration="3.404607032s" totalImagesPullingTime="1.313163402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:16.389697757 +0000 UTC m=+292.372990731" watchObservedRunningTime="2026-07-20 19:32:16.404607032 +0000 UTC m=+292.387899987" Jul 20 19:32:16.782479 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.782389 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq"] Jul 20 19:32:16.784817 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.784796 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.787187 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.787164 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mvztd\"" Jul 20 19:32:16.788035 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.788007 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 19:32:16.788140 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.788012 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 20 19:32:16.788140 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.788013 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 20 19:32:16.796469 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.796446 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq"] Jul 20 19:32:16.851143 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.851109 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.851332 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.851177 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.851332 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.851297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.851443 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.851350 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kxdz\" (UniqueName: \"kubernetes.io/projected/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-kube-api-access-4kxdz\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.952783 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.952742 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.952970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.952808 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.952970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.952850 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kxdz\" (UniqueName: \"kubernetes.io/projected/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-kube-api-access-4kxdz\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.952970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.952906 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.953133 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:16.952967 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 20 19:32:16.953133 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:16.953034 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-tls podName:35b485a1-3d66-4c4c-8c7d-5644d47ac29f nodeName:}" failed. No retries permitted until 2026-07-20 19:32:17.45301382 +0000 UTC m=+293.436306758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-j4bpq" (UID: "35b485a1-3d66-4c4c-8c7d-5644d47ac29f") : secret "prometheus-operator-tls" not found Jul 20 19:32:16.953886 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.953862 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.955589 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.955567 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:16.961257 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:16.961212 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kxdz\" (UniqueName: \"kubernetes.io/projected/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-kube-api-access-4kxdz\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:17.457654 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:17.457585 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:17.460511 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:17.460486 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/35b485a1-3d66-4c4c-8c7d-5644d47ac29f-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-j4bpq\" (UID: \"35b485a1-3d66-4c4c-8c7d-5644d47ac29f\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:17.697043 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:17.697008 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" Jul 20 19:32:17.839207 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:17.839186 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq"] Jul 20 19:32:17.841623 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:17.841579 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b485a1_3d66_4c4c_8c7d_5644d47ac29f.slice/crio-50d12eb163f0a72b45775c5a8b3707d92cd70773ad3591b7c58ee01abd9d7f3c WatchSource:0}: Error finding container 50d12eb163f0a72b45775c5a8b3707d92cd70773ad3591b7c58ee01abd9d7f3c: Status 404 returned error can't find the container with id 50d12eb163f0a72b45775c5a8b3707d92cd70773ad3591b7c58ee01abd9d7f3c Jul 20 19:32:18.382730 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:18.382694 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" event={"ID":"35b485a1-3d66-4c4c-8c7d-5644d47ac29f","Type":"ContainerStarted","Data":"50d12eb163f0a72b45775c5a8b3707d92cd70773ad3591b7c58ee01abd9d7f3c"} Jul 20 19:32:18.531976 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:18.531933 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-qndtv" podUID="d23a8cc4-8443-4595-9cf4-5dcacf92cc3f" Jul 20 19:32:21.396213 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:21.396133 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czg79" event={"ID":"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615","Type":"ContainerStarted","Data":"233dac81ba63eae27d5927d6389ba062cac3d871209a4e45678580946efc5a7e"} Jul 20 19:32:21.398800 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:21.398761 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" event={"ID":"35b485a1-3d66-4c4c-8c7d-5644d47ac29f","Type":"ContainerStarted","Data":"4de6f291de7c604fc380ae63a0505266dbc592494519dedc4fc114768dd6a305"} Jul 20 19:32:21.398931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:21.398817 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" event={"ID":"35b485a1-3d66-4c4c-8c7d-5644d47ac29f","Type":"ContainerStarted","Data":"c70d0ee042eec0f6c2ef1a23b2eb968068f9a3c95b430a032119f9a7273b90b0"} Jul 20 19:32:21.418959 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:21.418905 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j4bpq" podStartSLOduration=2.399707425 podStartE2EDuration="5.418884018s" totalImagesPullingTime="3.019176593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:21.417798148 +0000 UTC m=+297.401091125" watchObservedRunningTime="2026-07-20 19:32:21.418884018 +0000 UTC m=+297.402176978" Jul 20 19:32:22.358987 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:22.358958 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6gzpv" Jul 20 19:32:23.149382 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.147968 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw"] Jul 20 19:32:23.162707 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.162679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.166604 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.166577 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw"] Jul 20 19:32:23.169175 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.168952 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cwh6c\"" Jul 20 19:32:23.169347 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.169205 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 20 19:32:23.169604 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.169573 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 20 19:32:23.194432 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.194404 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hjqss"] Jul 20 19:32:23.198104 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.198081 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.200069 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.200048 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 19:32:23.200696 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.200677 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 19:32:23.200782 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.200764 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r5lrj\"" Jul 20 19:32:23.201020 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.200938 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 19:32:23.308363 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308330 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8xm6\" (UniqueName: \"kubernetes.io/projected/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-kube-api-access-q8xm6\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308363 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-root\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308574 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308393 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-wtmp\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308574 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308483 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-metrics-client-ca\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308574 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308528 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxvgv\" (UniqueName: \"kubernetes.io/projected/f8d8198b-bd86-41fd-9306-0b59259e2d7d-kube-api-access-cxvgv\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.308574 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308561 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-tls\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308780 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308630 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8d8198b-bd86-41fd-9306-0b59259e2d7d-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.308780 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308667 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8d8198b-bd86-41fd-9306-0b59259e2d7d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.308780 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308686 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308904 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308781 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-sys\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308904 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308818 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8d8198b-bd86-41fd-9306-0b59259e2d7d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.308904 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308860 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-textfile\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.308904 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.308897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.409660 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409553 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-tls\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.409660 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409607 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8d8198b-bd86-41fd-9306-0b59259e2d7d-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.409660 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409649 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8d8198b-bd86-41fd-9306-0b59259e2d7d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409688 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409740 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-sys\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:23.409750 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409783 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8d8198b-bd86-41fd-9306-0b59259e2d7d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:23.409827 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-tls podName:3d74cff2-d67a-498a-9c3d-fc196e32b6f1 nodeName:}" failed. No retries permitted until 2026-07-20 19:32:23.909806371 +0000 UTC m=+299.893099313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-tls") pod "node-exporter-hjqss" (UID: "3d74cff2-d67a-498a-9c3d-fc196e32b6f1") : secret "node-exporter-tls" not found Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-textfile\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409901 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.409953 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409936 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8xm6\" (UniqueName: \"kubernetes.io/projected/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-kube-api-access-q8xm6\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.410376 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409968 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-root\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.410376 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.409997 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-wtmp\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.410376 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.410029 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-metrics-client-ca\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.410376 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.410075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxvgv\" (UniqueName: \"kubernetes.io/projected/f8d8198b-bd86-41fd-9306-0b59259e2d7d-kube-api-access-cxvgv\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.410376 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.410363 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f8d8198b-bd86-41fd-9306-0b59259e2d7d-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.410665 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.410513 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-root\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.410665 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.410599 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-textfile\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.410665 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.410654 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-wtmp\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.411096 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.410996 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.411096 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.411043 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-sys\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.411324 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.411126 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-metrics-client-ca\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.413407 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.413378 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.413586 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.413570 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f8d8198b-bd86-41fd-9306-0b59259e2d7d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.413664 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.413597 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f8d8198b-bd86-41fd-9306-0b59259e2d7d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.422260 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.422237 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8xm6\" (UniqueName: \"kubernetes.io/projected/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-kube-api-access-q8xm6\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.423686 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.423667 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxvgv\" (UniqueName: \"kubernetes.io/projected/f8d8198b-bd86-41fd-9306-0b59259e2d7d-kube-api-access-cxvgv\") pod \"openshift-state-metrics-79f6958cdf-vdwhw\" (UID: \"f8d8198b-bd86-41fd-9306-0b59259e2d7d\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.473490 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.473457 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" Jul 20 19:32:23.914918 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.914877 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-tls\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:23.917556 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:23.917527 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3d74cff2-d67a-498a-9c3d-fc196e32b6f1-node-exporter-tls\") pod \"node-exporter-hjqss\" (UID: \"3d74cff2-d67a-498a-9c3d-fc196e32b6f1\") " pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:24.108251 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.108211 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hjqss" Jul 20 19:32:24.239673 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:24.239641 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d74cff2_d67a_498a_9c3d_fc196e32b6f1.slice/crio-8e8a331bdea32b0cda293d31348396fbf77efb1ef99246b9d96945a217e2c993 WatchSource:0}: Error finding container 8e8a331bdea32b0cda293d31348396fbf77efb1ef99246b9d96945a217e2c993: Status 404 returned error can't find the container with id 8e8a331bdea32b0cda293d31348396fbf77efb1ef99246b9d96945a217e2c993 Jul 20 19:32:24.253333 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.253301 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:32:24.256119 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.256089 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.258773 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.258990 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.259356 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.259580 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.259768 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.259917 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.260091 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.260257 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 19:32:24.261239 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.261221 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 19:32:24.261754 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.261630 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qjnqj\"" Jul 20 19:32:24.274066 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.273976 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:32:24.373157 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.373095 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw"] Jul 20 19:32:24.375158 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:24.375132 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d8198b_bd86_41fd_9306_0b59259e2d7d.slice/crio-605318c8adc1f624413e8e74398bd8e7a0fafcc30c8b26c29521769ce690e5f2 WatchSource:0}: Error finding container 605318c8adc1f624413e8e74398bd8e7a0fafcc30c8b26c29521769ce690e5f2: Status 404 returned error can't find the container with id 605318c8adc1f624413e8e74398bd8e7a0fafcc30c8b26c29521769ce690e5f2 Jul 20 19:32:24.412815 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.412789 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjqss" event={"ID":"3d74cff2-d67a-498a-9c3d-fc196e32b6f1","Type":"ContainerStarted","Data":"8e8a331bdea32b0cda293d31348396fbf77efb1ef99246b9d96945a217e2c993"} Jul 20 19:32:24.414256 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.414229 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" event={"ID":"f8d8198b-bd86-41fd-9306-0b59259e2d7d","Type":"ContainerStarted","Data":"605318c8adc1f624413e8e74398bd8e7a0fafcc30c8b26c29521769ce690e5f2"} Jul 20 19:32:24.416157 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.416131 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czg79" event={"ID":"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615","Type":"ContainerStarted","Data":"bc468fc51879d82e26198b1bf8f44856ec2c04ab10a20398125052d078b3b560"} Jul 20 19:32:24.419683 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419663 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.419768 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419697 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.419768 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419718 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.419768 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419745 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419785 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kphb\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-kube-api-access-2kphb\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419803 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-out\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419824 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419850 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419879 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419946 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-web-config\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.419981 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.420024 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.420242 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.420054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.450710 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.450674 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:32:24.451823 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.451804 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:32:24.464053 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.464033 2573 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 19:32:24.520823 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.520800 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kphb\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-kube-api-access-2kphb\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.520920 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.520831 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-out\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.520920 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.520849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521067 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521040 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521146 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521104 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521211 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521152 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-web-config\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521211 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521179 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521320 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521222 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521320 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521250 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521426 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521399 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521476 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521446 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521529 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521479 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521529 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.521658 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.521566 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.522206 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.522187 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.523292 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.523267 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-out\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.524187 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.523989 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 19:32:24.524187 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.523989 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 19:32:24.524187 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.524057 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 19:32:24.524187 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.523991 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 19:32:24.524494 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.524294 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 19:32:24.524494 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.524397 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 19:32:24.524494 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.524433 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 19:32:24.524494 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.524438 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 19:32:24.530048 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.529921 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kphb\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-kube-api-access-2kphb\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.530347 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.532025 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:24.532541 2573 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:24.532596 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls podName:3fc96e26-ec7c-4ea1-b64a-3676e0838d2d nodeName:}" failed. No retries permitted until 2026-07-20 19:32:25.032578738 +0000 UTC m=+301.015871679 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d") : secret "alertmanager-main-tls" not found Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.535275 2573 scope.go:117] "RemoveContainer" containerID="842e4992416f274bea1eacf72b1fa31ee4e58b46e33d571f1fcc7e2a6a5fb543" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.537328 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.537379 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.537554 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-web-config\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.537728 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.537995 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.538483 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:24.550130 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:24.540163 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:25.127703 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.127664 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:25.130724 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.130690 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:25.157799 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.157769 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-66657987f-6ljng"] Jul 20 19:32:25.161834 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.161802 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.166819 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.166793 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 20 19:32:25.166940 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.166854 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 20 19:32:25.167600 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.167456 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 20 19:32:25.167600 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.167491 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 20 19:32:25.167794 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.167736 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 20 19:32:25.168067 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.168048 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-641sfhmo97t0r\"" Jul 20 19:32:25.168294 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.168276 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5xnpw\"" Jul 20 19:32:25.172203 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.172176 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qjnqj\"" Jul 20 19:32:25.174182 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.174162 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66657987f-6ljng"] Jul 20 19:32:25.181321 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.181304 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:32:25.269091 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.269051 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-w6g6f"] Jul 20 19:32:25.272157 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.272091 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-w6g6f" Jul 20 19:32:25.275334 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.274738 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 19:32:25.275334 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.274947 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-xtzn5\"" Jul 20 19:32:25.275334 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.274987 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 19:32:25.286808 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.286753 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-w6g6f"] Jul 20 19:32:25.330380 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330351 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv8dd\" (UniqueName: \"kubernetes.io/projected/50fa9f60-7daa-44a7-a7ae-5eefb4921267-kube-api-access-nv8dd\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.330534 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.330534 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330467 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.330534 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330520 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.330740 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330571 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50fa9f60-7daa-44a7-a7ae-5eefb4921267-metrics-client-ca\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.330740 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330608 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-tls\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.330740 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-grpc-tls\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.330740 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.330684 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.424416 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.424382 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" event={"ID":"f8d8198b-bd86-41fd-9306-0b59259e2d7d","Type":"ContainerStarted","Data":"1b59833b405cebe44183d2ab9b71130d4e511421aac80429a372eb5ca96e9275"} Jul 20 19:32:25.424543 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.424450 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" event={"ID":"f8d8198b-bd86-41fd-9306-0b59259e2d7d","Type":"ContainerStarted","Data":"9b47836099b3a15d8f4ec2b117a76acc7aefd2cc9613fb65e1ebc6ce10f1ceed"} Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.429585 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.429663 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" event={"ID":"857f9a2d-27aa-4663-92bb-64406b0da2a8","Type":"ContainerStarted","Data":"801f95c7509fab88830f323cd0a4386eb2030aa49b005b6aa13fcde2ff0aa253"} Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.430845 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431206 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv8dd\" (UniqueName: \"kubernetes.io/projected/50fa9f60-7daa-44a7-a7ae-5eefb4921267-kube-api-access-nv8dd\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431251 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29qwt\" (UniqueName: \"kubernetes.io/projected/4b4e041a-e7a2-48d6-8a4d-2b011319fbd3-kube-api-access-29qwt\") pod \"downloads-5bc67dcd46-w6g6f\" (UID: \"4b4e041a-e7a2-48d6-8a4d-2b011319fbd3\") " pod="openshift-console/downloads-5bc67dcd46-w6g6f" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431342 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431367 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50fa9f60-7daa-44a7-a7ae-5eefb4921267-metrics-client-ca\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431436 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-tls\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431464 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-grpc-tls\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.431852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.431492 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.434282 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.434213 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/50fa9f60-7daa-44a7-a7ae-5eefb4921267-metrics-client-ca\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.436219 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.436171 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-tls\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.436712 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.436645 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.436934 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.436915 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.437370 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.437347 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.438139 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.438118 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.438220 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.438183 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/50fa9f60-7daa-44a7-a7ae-5eefb4921267-secret-grpc-tls\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.439602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.439575 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" Jul 20 19:32:25.440760 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.440739 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv8dd\" (UniqueName: \"kubernetes.io/projected/50fa9f60-7daa-44a7-a7ae-5eefb4921267-kube-api-access-nv8dd\") pod \"thanos-querier-66657987f-6ljng\" (UID: \"50fa9f60-7daa-44a7-a7ae-5eefb4921267\") " pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.470950 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.470894 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-pn76z" podStartSLOduration=43.799897777 podStartE2EDuration="46.470872148s" totalImagesPullingTime="2.670974371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:31:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:25.449439944 +0000 UTC m=+301.432732926" watchObservedRunningTime="2026-07-20 19:32:25.470872148 +0000 UTC m=+301.454165106" Jul 20 19:32:25.474079 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.474033 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:25.525843 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.525776 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:32:25.532422 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:25.532015 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc96e26_ec7c_4ea1_b64a_3676e0838d2d.slice/crio-72f29e947326439f03034728a9d32c4184dc372b5d875b3cd2e9e06d8d9cda08 WatchSource:0}: Error finding container 72f29e947326439f03034728a9d32c4184dc372b5d875b3cd2e9e06d8d9cda08: Status 404 returned error can't find the container with id 72f29e947326439f03034728a9d32c4184dc372b5d875b3cd2e9e06d8d9cda08 Jul 20 19:32:25.532422 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.532400 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29qwt\" (UniqueName: \"kubernetes.io/projected/4b4e041a-e7a2-48d6-8a4d-2b011319fbd3-kube-api-access-29qwt\") pod \"downloads-5bc67dcd46-w6g6f\" (UID: \"4b4e041a-e7a2-48d6-8a4d-2b011319fbd3\") " pod="openshift-console/downloads-5bc67dcd46-w6g6f" Jul 20 19:32:25.541635 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.538908 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 19:32:25.547284 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.547244 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29qwt\" (UniqueName: \"kubernetes.io/projected/4b4e041a-e7a2-48d6-8a4d-2b011319fbd3-kube-api-access-29qwt\") pod \"downloads-5bc67dcd46-w6g6f\" (UID: \"4b4e041a-e7a2-48d6-8a4d-2b011319fbd3\") " pod="openshift-console/downloads-5bc67dcd46-w6g6f" Jul 20 19:32:25.584365 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.584333 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-w6g6f" Jul 20 19:32:25.649152 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.648849 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-66657987f-6ljng"] Jul 20 19:32:25.654750 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:25.654719 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50fa9f60_7daa_44a7_a7ae_5eefb4921267.slice/crio-a323ae433c7decd3a23e364b381b8b73f4d317b26d5e24b202755b081f2406d0 WatchSource:0}: Error finding container a323ae433c7decd3a23e364b381b8b73f4d317b26d5e24b202755b081f2406d0: Status 404 returned error can't find the container with id a323ae433c7decd3a23e364b381b8b73f4d317b26d5e24b202755b081f2406d0 Jul 20 19:32:25.754695 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:25.754605 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-w6g6f"] Jul 20 19:32:25.760670 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:25.760640 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b4e041a_e7a2_48d6_8a4d_2b011319fbd3.slice/crio-cdeb0c256c41d3ba9b10ed25c353df5304775ba978f58f654030a96e30e91fef WatchSource:0}: Error finding container cdeb0c256c41d3ba9b10ed25c353df5304775ba978f58f654030a96e30e91fef: Status 404 returned error can't find the container with id cdeb0c256c41d3ba9b10ed25c353df5304775ba978f58f654030a96e30e91fef Jul 20 19:32:26.436750 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:26.436712 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerStarted","Data":"72f29e947326439f03034728a9d32c4184dc372b5d875b3cd2e9e06d8d9cda08"} Jul 20 19:32:26.438095 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:26.438065 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-w6g6f" event={"ID":"4b4e041a-e7a2-48d6-8a4d-2b011319fbd3","Type":"ContainerStarted","Data":"cdeb0c256c41d3ba9b10ed25c353df5304775ba978f58f654030a96e30e91fef"} Jul 20 19:32:26.441098 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:26.441072 2573 generic.go:356] "Generic (PLEG): container finished" podID="3d74cff2-d67a-498a-9c3d-fc196e32b6f1" containerID="c137274e4afcd0360eb7f340bdae86cb32ed125fa974621ec5aa2bc231f5324b" exitCode=0 Jul 20 19:32:26.441186 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:26.441167 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjqss" event={"ID":"3d74cff2-d67a-498a-9c3d-fc196e32b6f1","Type":"ContainerDied","Data":"c137274e4afcd0360eb7f340bdae86cb32ed125fa974621ec5aa2bc231f5324b"} Jul 20 19:32:26.443213 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:26.443111 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" event={"ID":"50fa9f60-7daa-44a7-a7ae-5eefb4921267","Type":"ContainerStarted","Data":"a323ae433c7decd3a23e364b381b8b73f4d317b26d5e24b202755b081f2406d0"} Jul 20 19:32:26.446403 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:26.446002 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" event={"ID":"f8d8198b-bd86-41fd-9306-0b59259e2d7d","Type":"ContainerStarted","Data":"9e695b9ae7bef5abb4ea71fc861a317ba2150c2c80e2ab2aa4f2fb79e8240f34"} Jul 20 19:32:27.457565 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.457529 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjqss" event={"ID":"3d74cff2-d67a-498a-9c3d-fc196e32b6f1","Type":"ContainerStarted","Data":"792bb297446f52f5efde08e30b08b03bd9ae027f3baf41fd1c158a90eef25183"} Jul 20 19:32:27.471667 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.471553 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vdwhw" podStartSLOduration=2.8504003239999998 podStartE2EDuration="4.471532793s" totalImagesPullingTime="1.621132469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:26.47701033 +0000 UTC m=+302.460303301" watchObservedRunningTime="2026-07-20 19:32:27.471532793 +0000 UTC m=+303.454825749" Jul 20 19:32:27.474331 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.473200 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5657dfbc9-7njq5"] Jul 20 19:32:27.486188 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.486161 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5657dfbc9-7njq5"] Jul 20 19:32:27.486335 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.486276 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.489662 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.489467 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-z56v6\"" Jul 20 19:32:27.489774 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.489746 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 19:32:27.490540 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.490088 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 20 19:32:27.490540 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.490136 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 20 19:32:27.490540 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.490096 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 20 19:32:27.490540 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.490420 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c73uvcjsbt0mo\"" Jul 20 19:32:27.653411 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.653376 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-secret-metrics-server-client-certs\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.653599 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.653559 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/910af318-ff0f-4185-8ba2-b1c64df3e05b-audit-log\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.653701 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.653602 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/910af318-ff0f-4185-8ba2-b1c64df3e05b-metrics-server-audit-profiles\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.653701 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.653651 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-secret-metrics-server-tls\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.653701 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.653685 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-client-ca-bundle\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.653818 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.653716 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/910af318-ff0f-4185-8ba2-b1c64df3e05b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.653818 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.653756 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j28rq\" (UniqueName: \"kubernetes.io/projected/910af318-ff0f-4185-8ba2-b1c64df3e05b-kube-api-access-j28rq\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.755937 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.755482 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/910af318-ff0f-4185-8ba2-b1c64df3e05b-audit-log\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.755937 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.755538 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/910af318-ff0f-4185-8ba2-b1c64df3e05b-metrics-server-audit-profiles\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.755937 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.755567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-secret-metrics-server-tls\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.755937 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.755597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-client-ca-bundle\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.755937 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.755637 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/910af318-ff0f-4185-8ba2-b1c64df3e05b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.755937 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.755660 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j28rq\" (UniqueName: \"kubernetes.io/projected/910af318-ff0f-4185-8ba2-b1c64df3e05b-kube-api-access-j28rq\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.755937 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.755698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-secret-metrics-server-client-certs\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.756408 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.756054 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/910af318-ff0f-4185-8ba2-b1c64df3e05b-audit-log\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.756812 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.756764 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/910af318-ff0f-4185-8ba2-b1c64df3e05b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.757137 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.757087 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/910af318-ff0f-4185-8ba2-b1c64df3e05b-metrics-server-audit-profiles\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.759761 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.759715 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-secret-metrics-server-tls\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.759761 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.759715 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-client-ca-bundle\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.760772 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.760726 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/910af318-ff0f-4185-8ba2-b1c64df3e05b-secret-metrics-server-client-certs\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.764427 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.764403 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j28rq\" (UniqueName: \"kubernetes.io/projected/910af318-ff0f-4185-8ba2-b1c64df3e05b-kube-api-access-j28rq\") pod \"metrics-server-5657dfbc9-7njq5\" (UID: \"910af318-ff0f-4185-8ba2-b1c64df3e05b\") " pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:27.800592 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:27.800551 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:28.398459 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.398422 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-74c76db8fb-7zppm"] Jul 20 19:32:28.408967 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.408937 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.410989 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.410960 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 20 19:32:28.411665 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.411603 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 20 19:32:28.411852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.411833 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 20 19:32:28.412323 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.412164 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2x6z4\"" Jul 20 19:32:28.412323 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.412195 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 20 19:32:28.412323 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.412266 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 20 19:32:28.417361 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.417296 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74c76db8fb-7zppm"] Jul 20 19:32:28.417463 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.417403 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 20 19:32:28.566054 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566005 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.566481 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566098 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-metrics-client-ca\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.566481 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566135 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-telemeter-client-tls\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.566481 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566173 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xng57\" (UniqueName: \"kubernetes.io/projected/2e626207-ab6d-4ccf-ba7f-da31812687e4-kube-api-access-xng57\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.566481 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566210 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-federate-client-tls\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.566481 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-serving-certs-ca-bundle\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.566481 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.566481 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.566381 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-secret-telemeter-client\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.666976 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-serving-certs-ca-bundle\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667036 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-secret-telemeter-client\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667325 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667110 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667325 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667294 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-metrics-client-ca\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667431 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667336 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-telemeter-client-tls\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667431 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667362 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xng57\" (UniqueName: \"kubernetes.io/projected/2e626207-ab6d-4ccf-ba7f-da31812687e4-kube-api-access-xng57\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667431 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667394 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-federate-client-tls\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.667888 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.667836 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-serving-certs-ca-bundle\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.668094 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.668064 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.668232 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.668199 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e626207-ab6d-4ccf-ba7f-da31812687e4-metrics-client-ca\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.670074 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.670015 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.670315 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.670294 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-telemeter-client-tls\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.670562 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.670537 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-secret-telemeter-client\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.670637 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.670546 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2e626207-ab6d-4ccf-ba7f-da31812687e4-federate-client-tls\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.675333 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.675309 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xng57\" (UniqueName: \"kubernetes.io/projected/2e626207-ab6d-4ccf-ba7f-da31812687e4-kube-api-access-xng57\") pod \"telemeter-client-74c76db8fb-7zppm\" (UID: \"2e626207-ab6d-4ccf-ba7f-da31812687e4\") " pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.723059 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.723026 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" Jul 20 19:32:28.928366 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.928269 2573 patch_prober.go:28] interesting pod/image-registry-6b54cbdc8b-htc59 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 20 19:32:28.928366 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:28.928344 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 19:32:29.530935 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:29.530894 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:32:29.868761 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:29.868265 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-74c76db8fb-7zppm"] Jul 20 19:32:29.876567 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:29.876541 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5657dfbc9-7njq5"] Jul 20 19:32:29.890597 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:29.890572 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b54cbdc8b-htc59"] Jul 20 19:32:29.891924 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:29.891897 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e626207_ab6d_4ccf_ba7f_da31812687e4.slice/crio-3cd2be39440f085cf79820584aaa8ef30b5b6f3ffd316a4e5abe242b2aaa3ce2 WatchSource:0}: Error finding container 3cd2be39440f085cf79820584aaa8ef30b5b6f3ffd316a4e5abe242b2aaa3ce2: Status 404 returned error can't find the container with id 3cd2be39440f085cf79820584aaa8ef30b5b6f3ffd316a4e5abe242b2aaa3ce2 Jul 20 19:32:29.897519 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:29.897462 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:30.484919 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.484849 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" event={"ID":"910af318-ff0f-4185-8ba2-b1c64df3e05b","Type":"ContainerStarted","Data":"059edf5068c33d528be54813544131b15bea9f14d088291703192c01af55c1d8"} Jul 20 19:32:30.488014 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.487942 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-czg79" event={"ID":"6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615","Type":"ContainerStarted","Data":"e87b8ca7833abac70517ddac346a73afeda69af5aff1f647b375504ef768e8bd"} Jul 20 19:32:30.490401 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.490368 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hjqss" event={"ID":"3d74cff2-d67a-498a-9c3d-fc196e32b6f1","Type":"ContainerStarted","Data":"af98a697152a49e30b9e94f088efdc4d259b9fc85ae7b583755ec0d500c26505"} Jul 20 19:32:30.492448 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.492410 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" event={"ID":"2e626207-ab6d-4ccf-ba7f-da31812687e4","Type":"ContainerStarted","Data":"3cd2be39440f085cf79820584aaa8ef30b5b6f3ffd316a4e5abe242b2aaa3ce2"} Jul 20 19:32:30.495040 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.494878 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" event={"ID":"50fa9f60-7daa-44a7-a7ae-5eefb4921267","Type":"ContainerStarted","Data":"bbc5c85bdd72f859cfc89a46947cff044374b544fc061400240abd128db72358"} Jul 20 19:32:30.495040 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.494952 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" event={"ID":"50fa9f60-7daa-44a7-a7ae-5eefb4921267","Type":"ContainerStarted","Data":"c80938d12ebc54de426ba74239795b550812680a7f8d1e12ba0ab30ff5ef3bc9"} Jul 20 19:32:30.495040 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.494995 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" event={"ID":"50fa9f60-7daa-44a7-a7ae-5eefb4921267","Type":"ContainerStarted","Data":"51823bb8fe68229b2566f9ad482c707e1b92ba6bfbe858ccd8a49752ee9f2687"} Jul 20 19:32:30.496944 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.496917 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerID="db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934" exitCode=0 Jul 20 19:32:30.497042 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.496949 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934"} Jul 20 19:32:30.508115 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.508067 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-czg79" podStartSLOduration=33.692908078 podStartE2EDuration="47.508047652s" totalImagesPullingTime="13.815139574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 19:31:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:30.506878837 +0000 UTC m=+306.490171793" watchObservedRunningTime="2026-07-20 19:32:30.508047652 +0000 UTC m=+306.491340607" Jul 20 19:32:30.526972 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:30.526621 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hjqss" podStartSLOduration=6.29940505 podStartE2EDuration="7.526596094s" totalImagesPullingTime="1.135444355s" totalInitContainerRuntime="91.746689ms" isStatefulPod=true podCreationTimestamp="2026-07-20 19:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:30.525076565 +0000 UTC m=+306.508369521" watchObservedRunningTime="2026-07-20 19:32:30.526596094 +0000 UTC m=+306.509889049" Jul 20 19:32:31.505327 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:31.505280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" event={"ID":"50fa9f60-7daa-44a7-a7ae-5eefb4921267","Type":"ContainerStarted","Data":"f483d4a662b5f8970a373621963b2c0d9a3e183289f1f989921b03e9a3ca2477"} Jul 20 19:32:31.505806 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:31.505409 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" event={"ID":"50fa9f60-7daa-44a7-a7ae-5eefb4921267","Type":"ContainerStarted","Data":"bc917344cb40df4f17a18208866fa716a38d0fe230ea9ee4404cd534b4843ea6"} Jul 20 19:32:31.506058 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:31.506029 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:31.506204 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:31.506146 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" event={"ID":"50fa9f60-7daa-44a7-a7ae-5eefb4921267","Type":"ContainerStarted","Data":"e72149c231751a5e51af7b112db4cfc7783a4783a4352b3a9c2e5b5f8b1ee3f0"} Jul 20 19:32:31.533235 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:31.533041 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" podStartSLOduration=1.521170976 podStartE2EDuration="6.533020963s" totalImagesPullingTime="5.011849987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:31.531841282 +0000 UTC m=+307.515134450" watchObservedRunningTime="2026-07-20 19:32:31.533020963 +0000 UTC m=+307.516313917" Jul 20 19:32:32.986801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.986776 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c4d49bd5b-92ldw"] Jul 20 19:32:32.991762 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.991703 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:32.994598 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.994562 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 19:32:32.994889 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.994868 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 19:32:32.995023 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.994980 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dnpwh\"" Jul 20 19:32:32.995112 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.995093 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 19:32:32.995167 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.995110 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 19:32:32.995329 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:32.995316 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 19:32:33.000745 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.000251 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c4d49bd5b-92ldw"] Jul 20 19:32:33.010036 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.009969 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-oauth-config\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.010036 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.010024 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-serving-cert\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.010205 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.010106 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-service-ca\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.010205 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.010134 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9z4h\" (UniqueName: \"kubernetes.io/projected/ef89f607-2820-4b85-85b0-1d3717228737-kube-api-access-h9z4h\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.010205 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.010183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-console-config\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.010349 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.010226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-oauth-serving-cert\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.110633 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.110572 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-console-config\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.110756 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.110648 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-oauth-serving-cert\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.110756 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.110678 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-oauth-config\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.110756 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.110725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-serving-cert\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.110924 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.110791 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-service-ca\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.110924 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.110817 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9z4h\" (UniqueName: \"kubernetes.io/projected/ef89f607-2820-4b85-85b0-1d3717228737-kube-api-access-h9z4h\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.111836 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.111775 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-service-ca\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.112148 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.112084 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-console-config\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.112148 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.112144 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-oauth-serving-cert\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.114190 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.114149 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-serving-cert\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.115262 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.115244 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-oauth-config\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.119018 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.118976 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9z4h\" (UniqueName: \"kubernetes.io/projected/ef89f607-2820-4b85-85b0-1d3717228737-kube-api-access-h9z4h\") pod \"console-c4d49bd5b-92ldw\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.309841 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.309807 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:33.469471 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.469216 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c4d49bd5b-92ldw"] Jul 20 19:32:33.471687 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:32:33.471658 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef89f607_2820_4b85_85b0_1d3717228737.slice/crio-2159d65ef7076ff8cdc9b7e969de3375ea7447f1037550ea794be9c62c03f75b WatchSource:0}: Error finding container 2159d65ef7076ff8cdc9b7e969de3375ea7447f1037550ea794be9c62c03f75b: Status 404 returned error can't find the container with id 2159d65ef7076ff8cdc9b7e969de3375ea7447f1037550ea794be9c62c03f75b Jul 20 19:32:33.521024 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.520873 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerStarted","Data":"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba"} Jul 20 19:32:33.521024 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.520957 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerStarted","Data":"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76"} Jul 20 19:32:33.521024 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.521002 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerStarted","Data":"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672"} Jul 20 19:32:33.521305 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.521041 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerStarted","Data":"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7"} Jul 20 19:32:33.521305 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.521081 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerStarted","Data":"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7"} Jul 20 19:32:33.521305 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.521120 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerStarted","Data":"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2"} Jul 20 19:32:33.523043 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.522991 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" event={"ID":"910af318-ff0f-4185-8ba2-b1c64df3e05b","Type":"ContainerStarted","Data":"32beb1a6e81031cc339253d0c4026ea640808bb07c4a7ead5a478a16a8ad2063"} Jul 20 19:32:33.524426 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.524388 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4d49bd5b-92ldw" event={"ID":"ef89f607-2820-4b85-85b0-1d3717228737","Type":"ContainerStarted","Data":"2159d65ef7076ff8cdc9b7e969de3375ea7447f1037550ea794be9c62c03f75b"} Jul 20 19:32:33.532692 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.532652 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" event={"ID":"2e626207-ab6d-4ccf-ba7f-da31812687e4","Type":"ContainerStarted","Data":"c7c2c5305897fa17c95b3b51bee3c549d5b69be4e269635f93b94593a13e0190"} Jul 20 19:32:33.532808 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.532751 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" event={"ID":"2e626207-ab6d-4ccf-ba7f-da31812687e4","Type":"ContainerStarted","Data":"720368354ffcb7dddc7b8e5a080edd86b03f79b3758de9fb9b4f1d694bdeb7a1"} Jul 20 19:32:33.532808 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.532775 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" event={"ID":"2e626207-ab6d-4ccf-ba7f-da31812687e4","Type":"ContainerStarted","Data":"baeccfc7d4ae8f1ed158769e4a6714ed3b74d284598cbd0fdd4511417ffe25b0"} Jul 20 19:32:33.548441 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.548312 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.157780499 podStartE2EDuration="9.5482963s" totalImagesPullingTime="6.371837532s" totalInitContainerRuntime="18.678269ms" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:33.547502111 +0000 UTC m=+309.530795095" watchObservedRunningTime="2026-07-20 19:32:33.5482963 +0000 UTC m=+309.531589257" Jul 20 19:32:33.566394 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.566342 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" podStartSLOduration=3.710304975 podStartE2EDuration="6.566324153s" totalImagesPullingTime="2.856019178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:33.565331385 +0000 UTC m=+309.548624344" watchObservedRunningTime="2026-07-20 19:32:33.566324153 +0000 UTC m=+309.549617111" Jul 20 19:32:33.586400 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:33.586343 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-74c76db8fb-7zppm" podStartSLOduration=2.727359633 podStartE2EDuration="5.586323634s" totalImagesPullingTime="2.858964001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:33.584591168 +0000 UTC m=+309.567884124" watchObservedRunningTime="2026-07-20 19:32:33.586323634 +0000 UTC m=+309.569616591" Jul 20 19:32:37.559559 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:37.559460 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4d49bd5b-92ldw" event={"ID":"ef89f607-2820-4b85-85b0-1d3717228737","Type":"ContainerStarted","Data":"890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c"} Jul 20 19:32:37.577692 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:37.577648 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c4d49bd5b-92ldw" podStartSLOduration=1.8638797070000002 podStartE2EDuration="5.577630865s" totalImagesPullingTime="3.713751158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:37.576243667 +0000 UTC m=+313.559536647" watchObservedRunningTime="2026-07-20 19:32:37.577630865 +0000 UTC m=+313.560923814" Jul 20 19:32:38.539983 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:38.539953 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-66657987f-6ljng" Jul 20 19:32:43.310314 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:43.310279 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:43.310796 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:43.310594 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:43.316947 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:43.316923 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:43.594900 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:43.594810 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:32:46.606466 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:46.606379 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-w6g6f" event={"ID":"4b4e041a-e7a2-48d6-8a4d-2b011319fbd3","Type":"ContainerStarted","Data":"e572d431a9843747a000d317470cacace0db15eacf95e620205859bfbd6ebea5"} Jul 20 19:32:46.606854 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:46.606542 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-w6g6f" Jul 20 19:32:46.608070 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:46.608046 2573 patch_prober.go:28] interesting pod/downloads-5bc67dcd46-w6g6f container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.22:8080/\": dial tcp 10.132.0.22:8080: connect: connection refused" start-of-body= Jul 20 19:32:46.608176 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:46.608100 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5bc67dcd46-w6g6f" podUID="4b4e041a-e7a2-48d6-8a4d-2b011319fbd3" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.22:8080/\": dial tcp 10.132.0.22:8080: connect: connection refused" Jul 20 19:32:46.624341 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:46.624298 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-w6g6f" podStartSLOduration=1.031576081 podStartE2EDuration="21.624287115s" totalImagesPullingTime="20.592711034s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:32:46.623563988 +0000 UTC m=+322.606856942" watchObservedRunningTime="2026-07-20 19:32:46.624287115 +0000 UTC m=+322.607580070" Jul 20 19:32:47.623844 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:47.623812 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-w6g6f" Jul 20 19:32:47.800932 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:47.800891 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:47.801104 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:47.800966 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:32:51.062488 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:51.062450 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c4d49bd5b-92ldw"] Jul 20 19:32:54.920713 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:54.920635 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" containerName="registry" containerID="cri-o://59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda" gracePeriod=30 Jul 20 19:32:55.254551 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.254527 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:55.342476 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.342436 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.342674 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.342511 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-trusted-ca\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.342674 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.342538 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-ca-trust-extracted\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.342674 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.342579 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-bound-sa-token\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.342970 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.342945 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-installation-pull-secrets\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.343631 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.342953 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:32:55.343631 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.343078 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-image-registry-private-configuration\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.343631 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.343111 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l4zd\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-kube-api-access-8l4zd\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.343631 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.343145 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-certificates\") pod \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\" (UID: \"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12\") " Jul 20 19:32:55.343631 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.343485 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-trusted-ca\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.343968 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.343820 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:32:55.345544 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.345513 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:32:55.345864 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.345538 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:32:55.345864 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.345657 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:32:55.346253 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.346229 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:32:55.346509 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.346482 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-kube-api-access-8l4zd" (OuterVolumeSpecName: "kube-api-access-8l4zd") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "kube-api-access-8l4zd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:32:55.352467 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.352443 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" (UID: "b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:32:55.444749 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.444716 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-bound-sa-token\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.444749 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.444743 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-installation-pull-secrets\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.444749 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.444753 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-image-registry-private-configuration\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.444972 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.444762 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l4zd\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-kube-api-access-8l4zd\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.444972 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.444771 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-certificates\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.444972 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.444781 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-registry-tls\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.444972 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.444789 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12-ca-trust-extracted\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:32:55.651341 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.651307 2573 generic.go:356] "Generic (PLEG): container finished" podID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" containerID="59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda" exitCode=0 Jul 20 19:32:55.651512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.651354 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" Jul 20 19:32:55.651512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.651369 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" event={"ID":"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12","Type":"ContainerDied","Data":"59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda"} Jul 20 19:32:55.651512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.651418 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b54cbdc8b-htc59" event={"ID":"b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12","Type":"ContainerDied","Data":"1b6cfd1d9844fff0c1c75ec7e7280567321ed126da3e9a9e5f9ee60cc55d7f16"} Jul 20 19:32:55.651512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.651439 2573 scope.go:117] "RemoveContainer" containerID="59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda" Jul 20 19:32:55.660767 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.660750 2573 scope.go:117] "RemoveContainer" containerID="59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda" Jul 20 19:32:55.661023 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:32:55.661005 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda\": container with ID starting with 59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda not found: ID does not exist" containerID="59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda" Jul 20 19:32:55.661091 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.661030 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda"} err="failed to get container status \"59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda\": rpc error: code = NotFound desc = could not find container \"59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda\": container with ID starting with 59f963fdc131aaf4615a8f0981e9438a9ddc332a9d054a8228aa926e14376eda not found: ID does not exist" Jul 20 19:32:55.673454 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.673427 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b54cbdc8b-htc59"] Jul 20 19:32:55.677800 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:55.677780 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b54cbdc8b-htc59"] Jul 20 19:32:56.539809 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:32:56.539179 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" path="/var/lib/kubelet/pods/b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12/volumes" Jul 20 19:33:03.699722 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:03.698928 2573 generic.go:356] "Generic (PLEG): container finished" podID="ba2123c0-fdb3-4b84-89fb-94a9467aa041" containerID="ac5997e63204fb6a59d8ef85c8a0a402324425dfab72108fec14600a682df077" exitCode=0 Jul 20 19:33:03.699722 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:03.699070 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" event={"ID":"ba2123c0-fdb3-4b84-89fb-94a9467aa041","Type":"ContainerDied","Data":"ac5997e63204fb6a59d8ef85c8a0a402324425dfab72108fec14600a682df077"} Jul 20 19:33:03.699722 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:03.699454 2573 scope.go:117] "RemoveContainer" containerID="ac5997e63204fb6a59d8ef85c8a0a402324425dfab72108fec14600a682df077" Jul 20 19:33:04.705047 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:04.705010 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-c5zqv" event={"ID":"ba2123c0-fdb3-4b84-89fb-94a9467aa041","Type":"ContainerStarted","Data":"c530b765d7d4f1ba02bbeeaaf458fc31c16e2e3a34ff9cb5abd19f2bd9e50428"} Jul 20 19:33:07.806687 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:07.806659 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:33:07.810554 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:07.810521 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5657dfbc9-7njq5" Jul 20 19:33:16.088852 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.088767 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c4d49bd5b-92ldw" podUID="ef89f607-2820-4b85-85b0-1d3717228737" containerName="console" containerID="cri-o://890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c" gracePeriod=15 Jul 20 19:33:16.364132 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.364109 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:33:16.436215 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436176 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-oauth-config\") pod \"ef89f607-2820-4b85-85b0-1d3717228737\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " Jul 20 19:33:16.436215 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436217 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-service-ca\") pod \"ef89f607-2820-4b85-85b0-1d3717228737\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " Jul 20 19:33:16.436462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436264 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-console-config\") pod \"ef89f607-2820-4b85-85b0-1d3717228737\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " Jul 20 19:33:16.436462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436313 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-oauth-serving-cert\") pod \"ef89f607-2820-4b85-85b0-1d3717228737\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " Jul 20 19:33:16.436462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436339 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-serving-cert\") pod \"ef89f607-2820-4b85-85b0-1d3717228737\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " Jul 20 19:33:16.436462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436396 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9z4h\" (UniqueName: \"kubernetes.io/projected/ef89f607-2820-4b85-85b0-1d3717228737-kube-api-access-h9z4h\") pod \"ef89f607-2820-4b85-85b0-1d3717228737\" (UID: \"ef89f607-2820-4b85-85b0-1d3717228737\") " Jul 20 19:33:16.436706 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436682 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-service-ca" (OuterVolumeSpecName: "service-ca") pod "ef89f607-2820-4b85-85b0-1d3717228737" (UID: "ef89f607-2820-4b85-85b0-1d3717228737"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:33:16.436802 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436770 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-console-config" (OuterVolumeSpecName: "console-config") pod "ef89f607-2820-4b85-85b0-1d3717228737" (UID: "ef89f607-2820-4b85-85b0-1d3717228737"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:33:16.436802 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.436783 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ef89f607-2820-4b85-85b0-1d3717228737" (UID: "ef89f607-2820-4b85-85b0-1d3717228737"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:33:16.438512 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.438492 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ef89f607-2820-4b85-85b0-1d3717228737" (UID: "ef89f607-2820-4b85-85b0-1d3717228737"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:16.438592 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.438534 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef89f607-2820-4b85-85b0-1d3717228737-kube-api-access-h9z4h" (OuterVolumeSpecName: "kube-api-access-h9z4h") pod "ef89f607-2820-4b85-85b0-1d3717228737" (UID: "ef89f607-2820-4b85-85b0-1d3717228737"). InnerVolumeSpecName "kube-api-access-h9z4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:33:16.438716 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.438591 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ef89f607-2820-4b85-85b0-1d3717228737" (UID: "ef89f607-2820-4b85-85b0-1d3717228737"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:16.537794 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.537761 2573 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-oauth-config\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:16.537794 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.537786 2573 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-service-ca\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:16.537794 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.537796 2573 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-console-config\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:16.538050 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.537804 2573 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef89f607-2820-4b85-85b0-1d3717228737-oauth-serving-cert\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:16.538050 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.537813 2573 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef89f607-2820-4b85-85b0-1d3717228737-console-serving-cert\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:16.538050 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.537823 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9z4h\" (UniqueName: \"kubernetes.io/projected/ef89f607-2820-4b85-85b0-1d3717228737-kube-api-access-h9z4h\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:16.764006 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.763966 2573 generic.go:356] "Generic (PLEG): container finished" podID="ef89f607-2820-4b85-85b0-1d3717228737" containerID="890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c" exitCode=0 Jul 20 19:33:16.764006 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.763997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4d49bd5b-92ldw" event={"ID":"ef89f607-2820-4b85-85b0-1d3717228737","Type":"ContainerDied","Data":"890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c"} Jul 20 19:33:16.764226 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.764013 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c4d49bd5b-92ldw" Jul 20 19:33:16.764226 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.764048 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c4d49bd5b-92ldw" event={"ID":"ef89f607-2820-4b85-85b0-1d3717228737","Type":"ContainerDied","Data":"2159d65ef7076ff8cdc9b7e969de3375ea7447f1037550ea794be9c62c03f75b"} Jul 20 19:33:16.764226 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.764069 2573 scope.go:117] "RemoveContainer" containerID="890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c" Jul 20 19:33:16.772714 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.772697 2573 scope.go:117] "RemoveContainer" containerID="890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c" Jul 20 19:33:16.772976 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:16.772959 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c\": container with ID starting with 890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c not found: ID does not exist" containerID="890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c" Jul 20 19:33:16.773026 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.772983 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c"} err="failed to get container status \"890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c\": rpc error: code = NotFound desc = could not find container \"890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c\": container with ID starting with 890fda080e8a4d90d6a669c937fd746e07dab022d4324caaccb2cb44805c274c not found: ID does not exist" Jul 20 19:33:16.783036 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.783008 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c4d49bd5b-92ldw"] Jul 20 19:33:16.788845 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:16.788821 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c4d49bd5b-92ldw"] Jul 20 19:33:18.536593 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:18.536559 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef89f607-2820-4b85-85b0-1d3717228737" path="/var/lib/kubelet/pods/ef89f607-2820-4b85-85b0-1d3717228737/volumes" Jul 20 19:33:38.431509 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:38.431462 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:33:38.433832 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:38.433807 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d23a8cc4-8443-4595-9cf4-5dcacf92cc3f-metrics-certs\") pod \"network-metrics-daemon-qndtv\" (UID: \"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f\") " pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:33:38.533398 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:38.533369 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pw7wf\"" Jul 20 19:33:38.541924 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:38.541901 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qndtv" Jul 20 19:33:38.672668 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:38.672644 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qndtv"] Jul 20 19:33:38.674910 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:33:38.674881 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23a8cc4_8443_4595_9cf4_5dcacf92cc3f.slice/crio-098e84c28b2374b218314c49b46a2c62760a44621715fb2069689230dff74940 WatchSource:0}: Error finding container 098e84c28b2374b218314c49b46a2c62760a44621715fb2069689230dff74940: Status 404 returned error can't find the container with id 098e84c28b2374b218314c49b46a2c62760a44621715fb2069689230dff74940 Jul 20 19:33:38.854778 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:38.854697 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qndtv" event={"ID":"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f","Type":"ContainerStarted","Data":"098e84c28b2374b218314c49b46a2c62760a44621715fb2069689230dff74940"} Jul 20 19:33:39.861128 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:39.861088 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qndtv" event={"ID":"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f","Type":"ContainerStarted","Data":"196c54f24b9175c25784cf056635c1636996abc048265a9eae6275e74bfce1cd"} Jul 20 19:33:39.861472 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:39.861155 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qndtv" event={"ID":"d23a8cc4-8443-4595-9cf4-5dcacf92cc3f","Type":"ContainerStarted","Data":"12631c0b9288b14e80bb92808308e90d650615bd4169b1ed9433a14936238e5b"} Jul 20 19:33:39.876391 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:39.876341 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qndtv" podStartSLOduration=374.887234525 podStartE2EDuration="6m15.876326352s" totalImagesPullingTime="989.091827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:33:39.87522854 +0000 UTC m=+375.858521508" watchObservedRunningTime="2026-07-20 19:33:39.876326352 +0000 UTC m=+375.859619307" Jul 20 19:33:43.538490 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.538458 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:33:43.538884 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.538736 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="alertmanager" containerID="cri-o://f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2" gracePeriod=120 Jul 20 19:33:43.538884 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.538791 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-metric" containerID="cri-o://b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76" gracePeriod=120 Jul 20 19:33:43.538884 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.538811 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-web" containerID="cri-o://30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7" gracePeriod=120 Jul 20 19:33:43.538884 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.538825 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy" containerID="cri-o://d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672" gracePeriod=120 Jul 20 19:33:43.538884 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.538844 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="config-reloader" containerID="cri-o://6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7" gracePeriod=120 Jul 20 19:33:43.539645 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.539584 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="prom-label-proxy" containerID="cri-o://1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba" gracePeriod=120 Jul 20 19:33:43.880054 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880017 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerID="1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba" exitCode=0 Jul 20 19:33:43.880054 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880045 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerID="d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672" exitCode=0 Jul 20 19:33:43.880054 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880054 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerID="6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7" exitCode=0 Jul 20 19:33:43.880337 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880062 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerID="f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2" exitCode=0 Jul 20 19:33:43.880337 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880092 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba"} Jul 20 19:33:43.880337 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880184 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672"} Jul 20 19:33:43.880337 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880210 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7"} Jul 20 19:33:43.880337 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:43.880235 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2"} Jul 20 19:33:44.812964 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.812935 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:44.887891 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.887856 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerID="b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76" exitCode=0 Jul 20 19:33:44.887891 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.887885 2573 generic.go:356] "Generic (PLEG): container finished" podID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerID="30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7" exitCode=0 Jul 20 19:33:44.888141 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.887993 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76"} Jul 20 19:33:44.888141 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.888013 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:44.888141 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.888091 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7"} Jul 20 19:33:44.888284 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.888143 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d","Type":"ContainerDied","Data":"72f29e947326439f03034728a9d32c4184dc372b5d875b3cd2e9e06d8d9cda08"} Jul 20 19:33:44.888284 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.888147 2573 scope.go:117] "RemoveContainer" containerID="1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba" Jul 20 19:33:44.896684 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.896662 2573 scope.go:117] "RemoveContainer" containerID="b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76" Jul 20 19:33:44.905693 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.905672 2573 scope.go:117] "RemoveContainer" containerID="d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672" Jul 20 19:33:44.912668 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.912652 2573 scope.go:117] "RemoveContainer" containerID="30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7" Jul 20 19:33:44.919102 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.919087 2573 scope.go:117] "RemoveContainer" containerID="6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7" Jul 20 19:33:44.925832 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.925814 2573 scope.go:117] "RemoveContainer" containerID="f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2" Jul 20 19:33:44.932450 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.932432 2573 scope.go:117] "RemoveContainer" containerID="db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934" Jul 20 19:33:44.939138 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.939121 2573 scope.go:117] "RemoveContainer" containerID="1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba" Jul 20 19:33:44.939379 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:44.939361 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba\": container with ID starting with 1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba not found: ID does not exist" containerID="1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba" Jul 20 19:33:44.939444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.939393 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba"} err="failed to get container status \"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba\": rpc error: code = NotFound desc = could not find container \"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba\": container with ID starting with 1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba not found: ID does not exist" Jul 20 19:33:44.939444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.939419 2573 scope.go:117] "RemoveContainer" containerID="b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76" Jul 20 19:33:44.939663 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:44.939646 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76\": container with ID starting with b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76 not found: ID does not exist" containerID="b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76" Jul 20 19:33:44.939709 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.939668 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76"} err="failed to get container status \"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76\": rpc error: code = NotFound desc = could not find container \"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76\": container with ID starting with b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76 not found: ID does not exist" Jul 20 19:33:44.939709 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.939683 2573 scope.go:117] "RemoveContainer" containerID="d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672" Jul 20 19:33:44.939881 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:44.939866 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672\": container with ID starting with d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672 not found: ID does not exist" containerID="d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672" Jul 20 19:33:44.939922 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.939885 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672"} err="failed to get container status \"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672\": rpc error: code = NotFound desc = could not find container \"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672\": container with ID starting with d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672 not found: ID does not exist" Jul 20 19:33:44.939922 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.939897 2573 scope.go:117] "RemoveContainer" containerID="30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7" Jul 20 19:33:44.940072 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:44.940059 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7\": container with ID starting with 30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7 not found: ID does not exist" containerID="30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7" Jul 20 19:33:44.940108 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940077 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7"} err="failed to get container status \"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7\": rpc error: code = NotFound desc = could not find container \"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7\": container with ID starting with 30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7 not found: ID does not exist" Jul 20 19:33:44.940108 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940093 2573 scope.go:117] "RemoveContainer" containerID="6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7" Jul 20 19:33:44.940316 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:44.940297 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7\": container with ID starting with 6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7 not found: ID does not exist" containerID="6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7" Jul 20 19:33:44.940359 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940321 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7"} err="failed to get container status \"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7\": rpc error: code = NotFound desc = could not find container \"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7\": container with ID starting with 6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7 not found: ID does not exist" Jul 20 19:33:44.940359 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940335 2573 scope.go:117] "RemoveContainer" containerID="f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2" Jul 20 19:33:44.940546 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:44.940529 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2\": container with ID starting with f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2 not found: ID does not exist" containerID="f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2" Jul 20 19:33:44.940625 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940554 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2"} err="failed to get container status \"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2\": rpc error: code = NotFound desc = could not find container \"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2\": container with ID starting with f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2 not found: ID does not exist" Jul 20 19:33:44.940625 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940575 2573 scope.go:117] "RemoveContainer" containerID="db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934" Jul 20 19:33:44.940817 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:33:44.940801 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934\": container with ID starting with db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934 not found: ID does not exist" containerID="db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934" Jul 20 19:33:44.940851 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940821 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934"} err="failed to get container status \"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934\": rpc error: code = NotFound desc = could not find container \"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934\": container with ID starting with db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934 not found: ID does not exist" Jul 20 19:33:44.940851 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.940835 2573 scope.go:117] "RemoveContainer" containerID="1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba" Jul 20 19:33:44.941049 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941031 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba"} err="failed to get container status \"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba\": rpc error: code = NotFound desc = could not find container \"1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba\": container with ID starting with 1b2fc49d6eac5d3240f53c63d1ec8553a45773f030a8b24a650abb64f31c6dba not found: ID does not exist" Jul 20 19:33:44.941148 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941051 2573 scope.go:117] "RemoveContainer" containerID="b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76" Jul 20 19:33:44.941245 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941230 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76"} err="failed to get container status \"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76\": rpc error: code = NotFound desc = could not find container \"b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76\": container with ID starting with b9c56373f09d4a4334fc0cd9b3d34d97c7f268c5a196ee6feafe9acbe3836e76 not found: ID does not exist" Jul 20 19:33:44.941290 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941245 2573 scope.go:117] "RemoveContainer" containerID="d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672" Jul 20 19:33:44.941420 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941402 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672"} err="failed to get container status \"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672\": rpc error: code = NotFound desc = could not find container \"d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672\": container with ID starting with d06edb06b5b17e214ed3701225a835570602bad04908a7c24683ec7f57afc672 not found: ID does not exist" Jul 20 19:33:44.941462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941421 2573 scope.go:117] "RemoveContainer" containerID="30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7" Jul 20 19:33:44.941634 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941599 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7"} err="failed to get container status \"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7\": rpc error: code = NotFound desc = could not find container \"30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7\": container with ID starting with 30d20dff59e8b26d10bdf9422dccb7368e2bb99fc43fad69c9f8733e3617f6f7 not found: ID does not exist" Jul 20 19:33:44.941692 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941635 2573 scope.go:117] "RemoveContainer" containerID="6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7" Jul 20 19:33:44.941874 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941856 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7"} err="failed to get container status \"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7\": rpc error: code = NotFound desc = could not find container \"6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7\": container with ID starting with 6ad1e3f42f30600e2b46b665300f97c40c7401ff256b51705ad7a62bc2248af7 not found: ID does not exist" Jul 20 19:33:44.941925 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.941874 2573 scope.go:117] "RemoveContainer" containerID="f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2" Jul 20 19:33:44.942082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.942066 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2"} err="failed to get container status \"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2\": rpc error: code = NotFound desc = could not find container \"f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2\": container with ID starting with f08520dc438fde81541837a136b71dcc823ad5f1bf0f5543bda48815cbd2b8b2 not found: ID does not exist" Jul 20 19:33:44.942082 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.942081 2573 scope.go:117] "RemoveContainer" containerID="db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934" Jul 20 19:33:44.942285 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.942268 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934"} err="failed to get container status \"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934\": rpc error: code = NotFound desc = could not find container \"db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934\": container with ID starting with db7ecfb86540fa29f42b2b180cc4ecdfee4dd138f3c8591670ad3dfb987ba934 not found: ID does not exist" Jul 20 19:33:44.989598 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989575 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-main-db\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989734 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989631 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-metrics-client-ca\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989734 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989659 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-tls-assets\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989734 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989700 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-web-config\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989734 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989730 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989763 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989791 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989824 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-web\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989861 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kphb\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-kube-api-access-2kphb\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989885 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-out\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989916 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-trusted-ca-bundle\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.989938 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989931 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-cluster-tls-config\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.990274 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989948 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:33:44.990274 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.989955 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-volume\") pod \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\" (UID: \"3fc96e26-ec7c-4ea1-b64a-3676e0838d2d\") " Jul 20 19:33:44.990274 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.990068 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:33:44.990428 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.990277 2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-metrics-client-ca\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:44.990428 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.990299 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-main-db\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:44.991192 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.991145 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 19:33:44.992459 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.992431 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:33:44.992593 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.992566 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:44.992904 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.992874 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-volume" (OuterVolumeSpecName: "config-volume") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:44.993071 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.993046 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:44.993315 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.993297 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:44.993391 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.993367 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-kube-api-access-2kphb" (OuterVolumeSpecName: "kube-api-access-2kphb") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "kube-api-access-2kphb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:33:44.994285 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.994261 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:44.994688 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.994670 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-out" (OuterVolumeSpecName: "config-out") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:33:44.998566 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:44.998543 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:45.005391 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.005370 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-web-config" (OuterVolumeSpecName: "web-config") pod "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" (UID: "3fc96e26-ec7c-4ea1-b64a-3676e0838d2d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:33:45.090844 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090796 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.090844 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090840 2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-cluster-tls-config\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.090844 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090850 2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-volume\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.090844 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090859 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-tls-assets\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.090844 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090867 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-web-config\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.091135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090874 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-main-tls\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.091135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090883 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.091135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090892 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.091135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090902 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.091135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090911 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kphb\" (UniqueName: \"kubernetes.io/projected/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-kube-api-access-2kphb\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.091135 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.090918 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d-config-out\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:33:45.214667 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.214637 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:33:45.219683 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.219660 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:33:45.245985 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.245956 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:33:45.246336 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246318 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="alertmanager" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246338 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="alertmanager" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246355 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-web" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246364 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-web" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246379 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" containerName="registry" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246387 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" containerName="registry" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246401 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="config-reloader" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246409 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="config-reloader" Jul 20 19:33:45.246418 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246418 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246426 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246437 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="init-config-reloader" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246446 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="init-config-reloader" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246456 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-metric" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246465 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-metric" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246475 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef89f607-2820-4b85-85b0-1d3717228737" containerName="console" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246484 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef89f607-2820-4b85-85b0-1d3717228737" containerName="console" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246498 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="prom-label-proxy" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246506 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="prom-label-proxy" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246583 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-web" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246596 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef89f607-2820-4b85-85b0-1d3717228737" containerName="console" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246606 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy-metric" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246632 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="alertmanager" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246639 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="kube-rbac-proxy" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246650 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3ffb3ec-d116-48ee-9c7f-f1d8b294ca12" containerName="registry" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246658 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="config-reloader" Jul 20 19:33:45.246814 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.246667 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" containerName="prom-label-proxy" Jul 20 19:33:45.250712 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.250694 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.252793 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.252762 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 20 19:33:45.252885 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.252824 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 20 19:33:45.252885 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.252837 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 20 19:33:45.252992 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.252923 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qjnqj\"" Jul 20 19:33:45.253137 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.253122 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 20 19:33:45.253207 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.253190 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 20 19:33:45.253281 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.253267 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 20 19:33:45.253333 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.253319 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 20 19:33:45.253462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.253447 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 20 19:33:45.259811 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.259682 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 20 19:33:45.263340 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.263319 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:33:45.393577 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-web-config\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393577 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393578 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5762909-8d95-4936-8af8-36b1f0480735-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393800 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393599 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-config-volume\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393800 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5762909-8d95-4936-8af8-36b1f0480735-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393800 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393744 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393800 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393767 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393807 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6sbk\" (UniqueName: \"kubernetes.io/projected/a5762909-8d95-4936-8af8-36b1f0480735-kube-api-access-w6sbk\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393834 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393857 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393876 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5762909-8d95-4936-8af8-36b1f0480735-config-out\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.393931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393890 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5762909-8d95-4936-8af8-36b1f0480735-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.394071 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393928 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.394071 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.393948 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5762909-8d95-4936-8af8-36b1f0480735-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494382 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494307 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-web-config\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494382 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494343 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5762909-8d95-4936-8af8-36b1f0480735-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494382 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494362 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-config-volume\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494492 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5762909-8d95-4936-8af8-36b1f0480735-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494533 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494559 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494602 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494591 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6sbk\" (UniqueName: \"kubernetes.io/projected/a5762909-8d95-4936-8af8-36b1f0480735-kube-api-access-w6sbk\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494646 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494670 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494701 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5762909-8d95-4936-8af8-36b1f0480735-config-out\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494726 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5762909-8d95-4936-8af8-36b1f0480735-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494749 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494771 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5762909-8d95-4936-8af8-36b1f0480735-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.494801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.494778 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a5762909-8d95-4936-8af8-36b1f0480735-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.495597 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.495568 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5762909-8d95-4936-8af8-36b1f0480735-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.497444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.497351 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a5762909-8d95-4936-8af8-36b1f0480735-config-out\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.497444 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.497439 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-config-volume\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.498170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.497761 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.498170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.497825 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.498170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.497894 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.498170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.497979 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-web-config\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.498170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.498118 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5762909-8d95-4936-8af8-36b1f0480735-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.498170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.498137 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.498170 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.498146 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a5762909-8d95-4936-8af8-36b1f0480735-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.499331 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.499316 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a5762909-8d95-4936-8af8-36b1f0480735-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.501817 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.501801 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6sbk\" (UniqueName: \"kubernetes.io/projected/a5762909-8d95-4936-8af8-36b1f0480735-kube-api-access-w6sbk\") pod \"alertmanager-main-0\" (UID: \"a5762909-8d95-4936-8af8-36b1f0480735\") " pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.562959 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.562931 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 20 19:33:45.699054 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.699029 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 20 19:33:45.701428 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:33:45.701397 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5762909_8d95_4936_8af8_36b1f0480735.slice/crio-b816359282606117ed4ccdd4157b9ec357670b178777e23dceb99523f5cf9c21 WatchSource:0}: Error finding container b816359282606117ed4ccdd4157b9ec357670b178777e23dceb99523f5cf9c21: Status 404 returned error can't find the container with id b816359282606117ed4ccdd4157b9ec357670b178777e23dceb99523f5cf9c21 Jul 20 19:33:45.893501 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.893468 2573 generic.go:356] "Generic (PLEG): container finished" podID="a5762909-8d95-4936-8af8-36b1f0480735" containerID="58fba29d35568fc64df387feac9cf46bdc1031447b53de3d51040e03750e3ec1" exitCode=0 Jul 20 19:33:45.893957 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.893557 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerDied","Data":"58fba29d35568fc64df387feac9cf46bdc1031447b53de3d51040e03750e3ec1"} Jul 20 19:33:45.893957 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:45.893667 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerStarted","Data":"b816359282606117ed4ccdd4157b9ec357670b178777e23dceb99523f5cf9c21"} Jul 20 19:33:46.536774 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.536738 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fc96e26-ec7c-4ea1-b64a-3676e0838d2d" path="/var/lib/kubelet/pods/3fc96e26-ec7c-4ea1-b64a-3676e0838d2d/volumes" Jul 20 19:33:46.903150 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.903112 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerStarted","Data":"0ddfd2206b065bb7c2cb05eab9685b2016e86a4255fba50d96995a2349be705b"} Jul 20 19:33:46.903508 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.903175 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerStarted","Data":"578b28a48a5853c0e9c962fccbec9c8ea45d00e1317aceac14c853290f614a2f"} Jul 20 19:33:46.903508 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.903200 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerStarted","Data":"a86b9971c98037c7864effb2abf4e5f91a3ea6efab6d4f14c61ec57b325e670b"} Jul 20 19:33:46.903508 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.903224 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerStarted","Data":"940bdd6eb9572a96151bfb59e44e3d2c6fb6d06f43fb6066b5fda4693153843c"} Jul 20 19:33:46.903508 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.903251 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerStarted","Data":"112b3b97d5e067c9d3db78d949bd8b6665e2e51fee7332aa999e096928c317a6"} Jul 20 19:33:46.903508 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.903274 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a5762909-8d95-4936-8af8-36b1f0480735","Type":"ContainerStarted","Data":"b9b9e43768bda6e17db76a43103850f92956dd1996f251b9a48b216df3c7fa21"} Jul 20 19:33:46.928203 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:33:46.928158 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.912813752 podStartE2EDuration="1.928143356s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.329604ms" isStatefulPod=false podCreationTimestamp="2026-07-20 19:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:33:46.926649867 +0000 UTC m=+382.909942821" watchObservedRunningTime="2026-07-20 19:33:46.928143356 +0000 UTC m=+382.911436310" Jul 20 19:35:23.378071 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.378041 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gn89v"] Jul 20 19:35:23.381186 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.381171 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.383584 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.383561 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-hr4zt\"" Jul 20 19:35:23.384331 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.384315 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 19:35:23.384384 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.384357 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 19:35:23.396968 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.396948 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gn89v"] Jul 20 19:35:23.495059 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.495018 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39f7306e-ef53-48ff-833a-bcd2981ec134-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gn89v\" (UID: \"39f7306e-ef53-48ff-833a-bcd2981ec134\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.495251 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.495139 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tr8z6\" (UniqueName: \"kubernetes.io/projected/39f7306e-ef53-48ff-833a-bcd2981ec134-kube-api-access-tr8z6\") pod \"cert-manager-cainjector-559cdd8c87-gn89v\" (UID: \"39f7306e-ef53-48ff-833a-bcd2981ec134\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.596560 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.596519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tr8z6\" (UniqueName: \"kubernetes.io/projected/39f7306e-ef53-48ff-833a-bcd2981ec134-kube-api-access-tr8z6\") pod \"cert-manager-cainjector-559cdd8c87-gn89v\" (UID: \"39f7306e-ef53-48ff-833a-bcd2981ec134\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.596783 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.596572 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39f7306e-ef53-48ff-833a-bcd2981ec134-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gn89v\" (UID: \"39f7306e-ef53-48ff-833a-bcd2981ec134\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.604508 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.604476 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39f7306e-ef53-48ff-833a-bcd2981ec134-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gn89v\" (UID: \"39f7306e-ef53-48ff-833a-bcd2981ec134\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.604745 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.604727 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tr8z6\" (UniqueName: \"kubernetes.io/projected/39f7306e-ef53-48ff-833a-bcd2981ec134-kube-api-access-tr8z6\") pod \"cert-manager-cainjector-559cdd8c87-gn89v\" (UID: \"39f7306e-ef53-48ff-833a-bcd2981ec134\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.703908 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.703835 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" Jul 20 19:35:23.835899 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:23.835791 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gn89v"] Jul 20 19:35:23.838366 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:35:23.838334 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39f7306e_ef53_48ff_833a_bcd2981ec134.slice/crio-1dbd5c56a2f60ca5be9b9159914985b2384c3740cb09b6d5a9b3ecea3eaae068 WatchSource:0}: Error finding container 1dbd5c56a2f60ca5be9b9159914985b2384c3740cb09b6d5a9b3ecea3eaae068: Status 404 returned error can't find the container with id 1dbd5c56a2f60ca5be9b9159914985b2384c3740cb09b6d5a9b3ecea3eaae068 Jul 20 19:35:24.296516 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:24.296479 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" event={"ID":"39f7306e-ef53-48ff-833a-bcd2981ec134","Type":"ContainerStarted","Data":"1dbd5c56a2f60ca5be9b9159914985b2384c3740cb09b6d5a9b3ecea3eaae068"} Jul 20 19:35:27.311942 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:27.311908 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" event={"ID":"39f7306e-ef53-48ff-833a-bcd2981ec134","Type":"ContainerStarted","Data":"a392a0fdb3a77daeb49682f27f1344572ed4261a6478cf9b1d55bfd42a576687"} Jul 20 19:35:27.327788 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:27.327740 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gn89v" podStartSLOduration=1.063350232 podStartE2EDuration="4.327725705s" totalImagesPullingTime="3.264375473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:35:27.326708162 +0000 UTC m=+483.310001119" watchObservedRunningTime="2026-07-20 19:35:27.327725705 +0000 UTC m=+483.311018660" Jul 20 19:35:28.650485 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.650449 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hs4lk"] Jul 20 19:35:28.653722 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.653696 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:28.655702 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.655684 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-b7gz9\"" Jul 20 19:35:28.663080 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.663056 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hs4lk"] Jul 20 19:35:28.740397 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.740357 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee53e275-b28e-47fd-9fa3-487a5ed4ba3b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hs4lk\" (UID: \"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:28.740541 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.740415 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww52l\" (UniqueName: \"kubernetes.io/projected/ee53e275-b28e-47fd-9fa3-487a5ed4ba3b-kube-api-access-ww52l\") pod \"cert-manager-webhook-6486ffd997-hs4lk\" (UID: \"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:28.841146 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.841112 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee53e275-b28e-47fd-9fa3-487a5ed4ba3b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hs4lk\" (UID: \"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:28.841303 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.841151 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww52l\" (UniqueName: \"kubernetes.io/projected/ee53e275-b28e-47fd-9fa3-487a5ed4ba3b-kube-api-access-ww52l\") pod \"cert-manager-webhook-6486ffd997-hs4lk\" (UID: \"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:28.852000 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.851970 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ee53e275-b28e-47fd-9fa3-487a5ed4ba3b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-hs4lk\" (UID: \"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:28.852140 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.852120 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww52l\" (UniqueName: \"kubernetes.io/projected/ee53e275-b28e-47fd-9fa3-487a5ed4ba3b-kube-api-access-ww52l\") pod \"cert-manager-webhook-6486ffd997-hs4lk\" (UID: \"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:28.962858 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:28.962774 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:29.096297 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:29.096271 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-hs4lk"] Jul 20 19:35:29.098823 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:35:29.098795 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee53e275_b28e_47fd_9fa3_487a5ed4ba3b.slice/crio-f12be3d27838e24beeeb8db3b8efa9a66160aeb92b812e762415ca551fd32d60 WatchSource:0}: Error finding container f12be3d27838e24beeeb8db3b8efa9a66160aeb92b812e762415ca551fd32d60: Status 404 returned error can't find the container with id f12be3d27838e24beeeb8db3b8efa9a66160aeb92b812e762415ca551fd32d60 Jul 20 19:35:29.321849 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:29.321758 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" event={"ID":"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b","Type":"ContainerStarted","Data":"5e7301c2b4ca3afde0f5063b6af2fb71edfca97f476e506db5f95159343b352f"} Jul 20 19:35:29.321849 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:29.321801 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" event={"ID":"ee53e275-b28e-47fd-9fa3-487a5ed4ba3b","Type":"ContainerStarted","Data":"f12be3d27838e24beeeb8db3b8efa9a66160aeb92b812e762415ca551fd32d60"} Jul 20 19:35:29.321849 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:29.321823 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:29.342510 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:29.342465 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" podStartSLOduration=1.342449125 podStartE2EDuration="1.342449125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:35:29.341184949 +0000 UTC m=+485.324477916" watchObservedRunningTime="2026-07-20 19:35:29.342449125 +0000 UTC m=+485.325742080" Jul 20 19:35:35.327586 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:35.327554 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-hs4lk" Jul 20 19:35:50.496300 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.496262 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw"] Jul 20 19:35:50.499644 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.499605 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.502449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.502427 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 19:35:50.502449 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.502442 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pg8q9\"" Jul 20 19:35:50.502672 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.502492 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 19:35:50.502808 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.502783 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 19:35:50.502917 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.502785 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 19:35:50.517238 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.517207 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw"] Jul 20 19:35:50.628017 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.627979 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-tmp\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.628176 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.628035 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-manifests\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.628176 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.628116 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2tvw\" (UniqueName: \"kubernetes.io/projected/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-kube-api-access-x2tvw\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.628256 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.628189 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-webhook-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.628256 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.628211 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.728931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.728898 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-tmp\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.729070 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.728947 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-manifests\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.729070 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.728987 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2tvw\" (UniqueName: \"kubernetes.io/projected/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-kube-api-access-x2tvw\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.729070 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.729044 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-webhook-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.729174 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.729072 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.729462 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.729441 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-manifests\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.731168 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.731145 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-tmp\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.732020 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.731999 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-apiservice-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.732116 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.732092 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-webhook-cert\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.736696 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.736676 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2tvw\" (UniqueName: \"kubernetes.io/projected/f8a4a6d4-2f96-4fe1-9264-cee91de1b178-kube-api-access-x2tvw\") pod \"opendatahub-operator-controller-manager-859c78d9fd-595mw\" (UID: \"f8a4a6d4-2f96-4fe1-9264-cee91de1b178\") " pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.810672 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.810573 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:50.949078 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:50.949051 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw"] Jul 20 19:35:50.952113 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:35:50.952083 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8a4a6d4_2f96_4fe1_9264_cee91de1b178.slice/crio-17b6d21fbd87bcf2141035e9f1123a52b4eed46d6569a26466f5faffffddda81 WatchSource:0}: Error finding container 17b6d21fbd87bcf2141035e9f1123a52b4eed46d6569a26466f5faffffddda81: Status 404 returned error can't find the container with id 17b6d21fbd87bcf2141035e9f1123a52b4eed46d6569a26466f5faffffddda81 Jul 20 19:35:51.425250 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:51.425209 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" event={"ID":"f8a4a6d4-2f96-4fe1-9264-cee91de1b178","Type":"ContainerStarted","Data":"17b6d21fbd87bcf2141035e9f1123a52b4eed46d6569a26466f5faffffddda81"} Jul 20 19:35:54.442754 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:54.442716 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" event={"ID":"f8a4a6d4-2f96-4fe1-9264-cee91de1b178","Type":"ContainerStarted","Data":"2869016f8e5443fedfd79d776ba170cc06d0ca2d9cfdd53cb941f2cfab08400a"} Jul 20 19:35:55.448890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:55.448806 2573 generic.go:356] "Generic (PLEG): container finished" podID="f8a4a6d4-2f96-4fe1-9264-cee91de1b178" containerID="2869016f8e5443fedfd79d776ba170cc06d0ca2d9cfdd53cb941f2cfab08400a" exitCode=0 Jul 20 19:35:55.448890 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:55.448872 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" event={"ID":"f8a4a6d4-2f96-4fe1-9264-cee91de1b178","Type":"ContainerDied","Data":"2869016f8e5443fedfd79d776ba170cc06d0ca2d9cfdd53cb941f2cfab08400a"} Jul 20 19:35:56.461599 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:56.461555 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" event={"ID":"f8a4a6d4-2f96-4fe1-9264-cee91de1b178","Type":"ContainerStarted","Data":"2601c6232cb7d1fcbdcd3c4d2f5a73d5f143f843062852c03660e5f0ebec605d"} Jul 20 19:35:56.462044 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:56.461764 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:35:56.490195 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:35:56.490144 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" podStartSLOduration=1.94957632 podStartE2EDuration="6.490129588s" totalImagesPullingTime="3.465496537s" totalInitContainerRuntime="1.075056731s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:35:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:35:56.488625285 +0000 UTC m=+512.471918252" watchObservedRunningTime="2026-07-20 19:35:56.490129588 +0000 UTC m=+512.473422542" Jul 20 19:36:07.468583 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:07.468547 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-859c78d9fd-595mw" Jul 20 19:36:15.595931 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.595894 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh"] Jul 20 19:36:15.602553 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.602532 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.604719 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.604700 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 19:36:15.604809 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.604720 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 20 19:36:15.605215 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.605170 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 20 19:36:15.605407 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.605394 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 19:36:15.623870 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.623843 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh"] Jul 20 19:36:15.641162 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.641130 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmcgg\" (UniqueName: \"kubernetes.io/projected/95140f23-e812-4918-b33f-69688ffdf031-kube-api-access-rmcgg\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.641298 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.641177 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95140f23-e812-4918-b33f-69688ffdf031-tls-certs\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.641298 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.641207 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95140f23-e812-4918-b33f-69688ffdf031-tmp\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.742191 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.742153 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmcgg\" (UniqueName: \"kubernetes.io/projected/95140f23-e812-4918-b33f-69688ffdf031-kube-api-access-rmcgg\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.742191 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.742196 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95140f23-e812-4918-b33f-69688ffdf031-tls-certs\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.742425 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.742215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95140f23-e812-4918-b33f-69688ffdf031-tmp\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.744534 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.744504 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95140f23-e812-4918-b33f-69688ffdf031-tmp\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.744711 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.744694 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95140f23-e812-4918-b33f-69688ffdf031-tls-certs\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.749460 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.749436 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmcgg\" (UniqueName: \"kubernetes.io/projected/95140f23-e812-4918-b33f-69688ffdf031-kube-api-access-rmcgg\") pod \"kube-auth-proxy-86d68bcf69-7sfdh\" (UID: \"95140f23-e812-4918-b33f-69688ffdf031\") " pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:15.911958 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:15.911923 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" Jul 20 19:36:16.041845 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.041784 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh"] Jul 20 19:36:16.044189 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:36:16.044161 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95140f23_e812_4918_b33f_69688ffdf031.slice/crio-00079d35c58903b75f893c42bd41068334ac95bf4b9f210ad3d71fcb1380bbef WatchSource:0}: Error finding container 00079d35c58903b75f893c42bd41068334ac95bf4b9f210ad3d71fcb1380bbef: Status 404 returned error can't find the container with id 00079d35c58903b75f893c42bd41068334ac95bf4b9f210ad3d71fcb1380bbef Jul 20 19:36:16.562965 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.562926 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" event={"ID":"95140f23-e812-4918-b33f-69688ffdf031","Type":"ContainerStarted","Data":"00079d35c58903b75f893c42bd41068334ac95bf4b9f210ad3d71fcb1380bbef"} Jul 20 19:36:16.942141 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.942077 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx"] Jul 20 19:36:16.945192 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.945167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:16.947325 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.947302 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 19:36:16.948081 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.948055 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 19:36:16.948199 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.948138 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xtkz7\"" Jul 20 19:36:16.948913 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.948886 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 19:36:16.949020 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.948953 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 19:36:16.949875 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.949852 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 19:36:16.966849 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:16.966826 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx"] Jul 20 19:36:17.054738 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.054703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88c32b88-41a4-4dc8-b496-de6ea8a379ac-manager-config\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.054738 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.054740 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88c32b88-41a4-4dc8-b496-de6ea8a379ac-cert\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.054945 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.054774 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88c32b88-41a4-4dc8-b496-de6ea8a379ac-metrics-cert\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.054945 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.054801 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkxxw\" (UniqueName: \"kubernetes.io/projected/88c32b88-41a4-4dc8-b496-de6ea8a379ac-kube-api-access-pkxxw\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.156056 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.156014 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88c32b88-41a4-4dc8-b496-de6ea8a379ac-manager-config\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.156248 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.156069 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88c32b88-41a4-4dc8-b496-de6ea8a379ac-cert\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.156248 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.156205 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88c32b88-41a4-4dc8-b496-de6ea8a379ac-metrics-cert\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.156371 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.156274 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkxxw\" (UniqueName: \"kubernetes.io/projected/88c32b88-41a4-4dc8-b496-de6ea8a379ac-kube-api-access-pkxxw\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.156892 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.156849 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88c32b88-41a4-4dc8-b496-de6ea8a379ac-manager-config\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.158978 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.158947 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88c32b88-41a4-4dc8-b496-de6ea8a379ac-cert\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.159180 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.159127 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88c32b88-41a4-4dc8-b496-de6ea8a379ac-metrics-cert\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.177749 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.177718 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkxxw\" (UniqueName: \"kubernetes.io/projected/88c32b88-41a4-4dc8-b496-de6ea8a379ac-kube-api-access-pkxxw\") pod \"lws-controller-manager-65bc99bbc8-9qlhx\" (UID: \"88c32b88-41a4-4dc8-b496-de6ea8a379ac\") " pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.257110 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.257025 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:17.399114 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.399075 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx"] Jul 20 19:36:17.402903 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:36:17.402873 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c32b88_41a4_4dc8_b496_de6ea8a379ac.slice/crio-6025ac8447eff06647a56d889c5f24716be5d3c35606d7753aba0312e954db7f WatchSource:0}: Error finding container 6025ac8447eff06647a56d889c5f24716be5d3c35606d7753aba0312e954db7f: Status 404 returned error can't find the container with id 6025ac8447eff06647a56d889c5f24716be5d3c35606d7753aba0312e954db7f Jul 20 19:36:17.569806 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:17.569722 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" event={"ID":"88c32b88-41a4-4dc8-b496-de6ea8a379ac","Type":"ContainerStarted","Data":"6025ac8447eff06647a56d889c5f24716be5d3c35606d7753aba0312e954db7f"} Jul 20 19:36:20.587708 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:20.587674 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" event={"ID":"88c32b88-41a4-4dc8-b496-de6ea8a379ac","Type":"ContainerStarted","Data":"1853f0cbb19a37a18b5c76b1f4e703f08e6e0c2d497b16393b534b354691376a"} Jul 20 19:36:20.588151 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:20.587732 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:36:20.589161 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:20.589135 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" event={"ID":"95140f23-e812-4918-b33f-69688ffdf031","Type":"ContainerStarted","Data":"a5f463a937db6ceee66aec9da5b82822fbd36a2385a977aa11fbb156ce6e01c2"} Jul 20 19:36:20.604004 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:20.603962 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" podStartSLOduration=1.9590511130000001 podStartE2EDuration="4.603948659s" totalImagesPullingTime="2.644897546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:20.602742059 +0000 UTC m=+536.586035065" watchObservedRunningTime="2026-07-20 19:36:20.603948659 +0000 UTC m=+536.587241614" Jul 20 19:36:20.619048 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:20.619007 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86d68bcf69-7sfdh" podStartSLOduration=1.6833313479999998 podStartE2EDuration="5.618994599s" totalImagesPullingTime="3.935663251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:36:20.617267747 +0000 UTC m=+536.600560702" watchObservedRunningTime="2026-07-20 19:36:20.618994599 +0000 UTC m=+536.602287554" Jul 20 19:36:31.595750 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:36:31.595710 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65bc99bbc8-9qlhx" Jul 20 19:37:24.485764 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:24.485726 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:37:24.486409 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:24.485738 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:37:53.677778 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.677740 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn"] Jul 20 19:37:53.681125 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.681103 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" Jul 20 19:37:53.684832 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.684810 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-m6sx6\"" Jul 20 19:37:53.685323 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.685305 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 19:37:53.685405 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.685346 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 19:37:53.685564 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.685549 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 20 19:37:53.696903 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.696881 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn"] Jul 20 19:37:53.784594 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.784553 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z4xm\" (UniqueName: \"kubernetes.io/projected/247d3f3b-1054-433d-a9b9-fcaecfb6dba5-kube-api-access-7z4xm\") pod \"dns-operator-controller-manager-864dd78c4c-2xlxn\" (UID: \"247d3f3b-1054-433d-a9b9-fcaecfb6dba5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" Jul 20 19:37:53.885299 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.885267 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z4xm\" (UniqueName: \"kubernetes.io/projected/247d3f3b-1054-433d-a9b9-fcaecfb6dba5-kube-api-access-7z4xm\") pod \"dns-operator-controller-manager-864dd78c4c-2xlxn\" (UID: \"247d3f3b-1054-433d-a9b9-fcaecfb6dba5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" Jul 20 19:37:53.901300 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.901271 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z4xm\" (UniqueName: \"kubernetes.io/projected/247d3f3b-1054-433d-a9b9-fcaecfb6dba5-kube-api-access-7z4xm\") pod \"dns-operator-controller-manager-864dd78c4c-2xlxn\" (UID: \"247d3f3b-1054-433d-a9b9-fcaecfb6dba5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" Jul 20 19:37:53.991255 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:53.991187 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" Jul 20 19:37:54.131367 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:54.131343 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn"] Jul 20 19:37:54.133589 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:37:54.133566 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod247d3f3b_1054_433d_a9b9_fcaecfb6dba5.slice/crio-ed0b4055ed751fb61049c1aa4a1e155349feca9f6123e35ca819b2e604cfc7ee WatchSource:0}: Error finding container ed0b4055ed751fb61049c1aa4a1e155349feca9f6123e35ca819b2e604cfc7ee: Status 404 returned error can't find the container with id ed0b4055ed751fb61049c1aa4a1e155349feca9f6123e35ca819b2e604cfc7ee Jul 20 19:37:54.135562 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:54.135545 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 19:37:55.030050 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:55.030001 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" event={"ID":"247d3f3b-1054-433d-a9b9-fcaecfb6dba5","Type":"ContainerStarted","Data":"ed0b4055ed751fb61049c1aa4a1e155349feca9f6123e35ca819b2e604cfc7ee"} Jul 20 19:37:57.040817 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.040771 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" event={"ID":"247d3f3b-1054-433d-a9b9-fcaecfb6dba5","Type":"ContainerStarted","Data":"b2ec345fa33ee20df11e7174e4969bb52e947c9b9775c169d4e57962f2b01518"} Jul 20 19:37:57.041287 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.040864 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" Jul 20 19:37:57.266936 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.266887 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" podStartSLOduration=2.113755378 podStartE2EDuration="4.266868553s" totalImagesPullingTime="2.153113175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:37:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:37:57.079360489 +0000 UTC m=+633.062653457" watchObservedRunningTime="2026-07-20 19:37:57.266868553 +0000 UTC m=+633.250161508" Jul 20 19:37:57.267102 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.267007 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr"] Jul 20 19:37:57.270536 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.270518 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.272963 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.272940 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 20 19:37:57.273061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.273021 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 20 19:37:57.273061 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.273038 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5vdsb\"" Jul 20 19:37:57.290639 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.290590 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr"] Jul 20 19:37:57.318399 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.318340 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af210c1-043f-4d2c-aed6-50395ddb59cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.318510 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.318402 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8af210c1-043f-4d2c-aed6-50395ddb59cd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.318510 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.318472 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s958q\" (UniqueName: \"kubernetes.io/projected/8af210c1-043f-4d2c-aed6-50395ddb59cd-kube-api-access-s958q\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.419568 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.419530 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8af210c1-043f-4d2c-aed6-50395ddb59cd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.419752 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.419588 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s958q\" (UniqueName: \"kubernetes.io/projected/8af210c1-043f-4d2c-aed6-50395ddb59cd-kube-api-access-s958q\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.419752 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.419659 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af210c1-043f-4d2c-aed6-50395ddb59cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.419872 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:37:57.419819 2573 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 20 19:37:57.419923 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:37:57.419895 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8af210c1-043f-4d2c-aed6-50395ddb59cd-plugin-serving-cert podName:8af210c1-043f-4d2c-aed6-50395ddb59cd nodeName:}" failed. No retries permitted until 2026-07-20 19:37:57.919872313 +0000 UTC m=+633.903165471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/8af210c1-043f-4d2c-aed6-50395ddb59cd-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-8ksdr" (UID: "8af210c1-043f-4d2c-aed6-50395ddb59cd") : secret "plugin-serving-cert" not found Jul 20 19:37:57.420308 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.420282 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8af210c1-043f-4d2c-aed6-50395ddb59cd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.434606 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.434578 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s958q\" (UniqueName: \"kubernetes.io/projected/8af210c1-043f-4d2c-aed6-50395ddb59cd-kube-api-access-s958q\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.923383 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.923348 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af210c1-043f-4d2c-aed6-50395ddb59cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:57.925671 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:57.925642 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af210c1-043f-4d2c-aed6-50395ddb59cd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8ksdr\" (UID: \"8af210c1-043f-4d2c-aed6-50395ddb59cd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:58.180203 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:58.180129 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" Jul 20 19:37:58.319972 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:58.319947 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr"] Jul 20 19:37:58.321949 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:37:58.321923 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8af210c1_043f_4d2c_aed6_50395ddb59cd.slice/crio-91803d90dab1969f4350629ee38d46fb368ce07e531cf387e1ccc6eaec8e1903 WatchSource:0}: Error finding container 91803d90dab1969f4350629ee38d46fb368ce07e531cf387e1ccc6eaec8e1903: Status 404 returned error can't find the container with id 91803d90dab1969f4350629ee38d46fb368ce07e531cf387e1ccc6eaec8e1903 Jul 20 19:37:59.054827 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:37:59.054788 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" event={"ID":"8af210c1-043f-4d2c-aed6-50395ddb59cd","Type":"ContainerStarted","Data":"91803d90dab1969f4350629ee38d46fb368ce07e531cf387e1ccc6eaec8e1903"} Jul 20 19:38:08.048351 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:08.048315 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-2xlxn" Jul 20 19:38:09.456925 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.456881 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5"] Jul 20 19:38:09.488740 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.488709 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5"] Jul 20 19:38:09.488936 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.488821 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:09.491488 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.491467 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7g7tv\"" Jul 20 19:38:09.639505 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.639464 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6bw9\" (UniqueName: \"kubernetes.io/projected/e70446cf-bcca-4823-9662-60025b2a14a3-kube-api-access-h6bw9\") pod \"kuadrant-operator-controller-manager-86878c769f-7p6d5\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:09.639722 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.639522 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70446cf-bcca-4823-9662-60025b2a14a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7p6d5\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:09.740580 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.740493 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6bw9\" (UniqueName: \"kubernetes.io/projected/e70446cf-bcca-4823-9662-60025b2a14a3-kube-api-access-h6bw9\") pod \"kuadrant-operator-controller-manager-86878c769f-7p6d5\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:09.740580 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.740555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70446cf-bcca-4823-9662-60025b2a14a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7p6d5\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:09.740978 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.740952 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70446cf-bcca-4823-9662-60025b2a14a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7p6d5\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:09.754023 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.753993 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6bw9\" (UniqueName: \"kubernetes.io/projected/e70446cf-bcca-4823-9662-60025b2a14a3-kube-api-access-h6bw9\") pod \"kuadrant-operator-controller-manager-86878c769f-7p6d5\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:09.801780 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:09.801748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:19.899065 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:19.899039 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5"] Jul 20 19:38:19.900603 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:38:19.900579 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode70446cf_bcca_4823_9662_60025b2a14a3.slice/crio-64a9f5a1b4c5f3e2f815d17a59dc6830de3ebf5a80be27cb329100e4fe2005c7 WatchSource:0}: Error finding container 64a9f5a1b4c5f3e2f815d17a59dc6830de3ebf5a80be27cb329100e4fe2005c7: Status 404 returned error can't find the container with id 64a9f5a1b4c5f3e2f815d17a59dc6830de3ebf5a80be27cb329100e4fe2005c7 Jul 20 19:38:20.173463 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:20.173432 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" event={"ID":"e70446cf-bcca-4823-9662-60025b2a14a3","Type":"ContainerStarted","Data":"64a9f5a1b4c5f3e2f815d17a59dc6830de3ebf5a80be27cb329100e4fe2005c7"} Jul 20 19:38:20.174773 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:20.174746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" event={"ID":"8af210c1-043f-4d2c-aed6-50395ddb59cd","Type":"ContainerStarted","Data":"ee90f5514fecc45c2fd3d85b8c2ecfc3687610682c8410f2fdec794027ffcd02"} Jul 20 19:38:20.194262 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:20.194212 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ksdr" podStartSLOduration=1.697289573 podStartE2EDuration="23.194196203s" totalImagesPullingTime="21.49690663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:37:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:38:20.192958714 +0000 UTC m=+656.176251669" watchObservedRunningTime="2026-07-20 19:38:20.194196203 +0000 UTC m=+656.177489160" Jul 20 19:38:25.205546 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:25.205506 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" event={"ID":"e70446cf-bcca-4823-9662-60025b2a14a3","Type":"ContainerStarted","Data":"83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1"} Jul 20 19:38:25.206026 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:25.205579 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:25.231840 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:25.231791 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" podStartSLOduration=11.411858108 podStartE2EDuration="16.231775166s" totalImagesPullingTime="4.819917058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:38:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:38:25.229787591 +0000 UTC m=+661.213080556" watchObservedRunningTime="2026-07-20 19:38:25.231775166 +0000 UTC m=+661.215068186" Jul 20 19:38:36.212441 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:36.212409 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:38:56.326497 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.326462 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-zzssr"] Jul 20 19:38:56.401883 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.401845 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zzssr"] Jul 20 19:38:56.402078 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.401963 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zzssr" Jul 20 19:38:56.404037 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.404011 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2qtj\"" Jul 20 19:38:56.555538 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.555503 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw8jp\" (UniqueName: \"kubernetes.io/projected/78b542af-05ce-45a0-bad7-b7152db8b872-kube-api-access-fw8jp\") pod \"authorino-58cd955675-zzssr\" (UID: \"78b542af-05ce-45a0-bad7-b7152db8b872\") " pod="kuadrant-system/authorino-58cd955675-zzssr" Jul 20 19:38:56.656881 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.656830 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw8jp\" (UniqueName: \"kubernetes.io/projected/78b542af-05ce-45a0-bad7-b7152db8b872-kube-api-access-fw8jp\") pod \"authorino-58cd955675-zzssr\" (UID: \"78b542af-05ce-45a0-bad7-b7152db8b872\") " pod="kuadrant-system/authorino-58cd955675-zzssr" Jul 20 19:38:56.665968 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.665939 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw8jp\" (UniqueName: \"kubernetes.io/projected/78b542af-05ce-45a0-bad7-b7152db8b872-kube-api-access-fw8jp\") pod \"authorino-58cd955675-zzssr\" (UID: \"78b542af-05ce-45a0-bad7-b7152db8b872\") " pod="kuadrant-system/authorino-58cd955675-zzssr" Jul 20 19:38:56.711421 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.711393 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zzssr" Jul 20 19:38:56.839817 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:56.839792 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zzssr"] Jul 20 19:38:56.842604 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:38:56.842578 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78b542af_05ce_45a0_bad7_b7152db8b872.slice/crio-63ac730f913da01f49d21d1ceb6cfd10273d9d5f731e8841e12185cde33c8192 WatchSource:0}: Error finding container 63ac730f913da01f49d21d1ceb6cfd10273d9d5f731e8841e12185cde33c8192: Status 404 returned error can't find the container with id 63ac730f913da01f49d21d1ceb6cfd10273d9d5f731e8841e12185cde33c8192 Jul 20 19:38:57.364923 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:38:57.364890 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zzssr" event={"ID":"78b542af-05ce-45a0-bad7-b7152db8b872","Type":"ContainerStarted","Data":"63ac730f913da01f49d21d1ceb6cfd10273d9d5f731e8841e12185cde33c8192"} Jul 20 19:39:00.384113 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:00.384074 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zzssr" event={"ID":"78b542af-05ce-45a0-bad7-b7152db8b872","Type":"ContainerStarted","Data":"7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2"} Jul 20 19:39:00.398828 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:00.398778 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-zzssr" podStartSLOduration=1.226480255 podStartE2EDuration="4.398763415s" totalImagesPullingTime="3.17228316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:38:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:39:00.397518955 +0000 UTC m=+696.380811908" watchObservedRunningTime="2026-07-20 19:39:00.398763415 +0000 UTC m=+696.382056373" Jul 20 19:39:00.919991 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:00.919943 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zzssr"] Jul 20 19:39:02.395131 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:02.395091 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-zzssr" podUID="78b542af-05ce-45a0-bad7-b7152db8b872" containerName="authorino" containerID="cri-o://7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2" gracePeriod=30 Jul 20 19:39:02.636376 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:02.636348 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zzssr" Jul 20 19:39:02.715872 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:02.715789 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw8jp\" (UniqueName: \"kubernetes.io/projected/78b542af-05ce-45a0-bad7-b7152db8b872-kube-api-access-fw8jp\") pod \"78b542af-05ce-45a0-bad7-b7152db8b872\" (UID: \"78b542af-05ce-45a0-bad7-b7152db8b872\") " Jul 20 19:39:02.717801 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:02.717769 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78b542af-05ce-45a0-bad7-b7152db8b872-kube-api-access-fw8jp" (OuterVolumeSpecName: "kube-api-access-fw8jp") pod "78b542af-05ce-45a0-bad7-b7152db8b872" (UID: "78b542af-05ce-45a0-bad7-b7152db8b872"). InnerVolumeSpecName "kube-api-access-fw8jp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:39:02.816974 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:02.816937 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw8jp\" (UniqueName: \"kubernetes.io/projected/78b542af-05ce-45a0-bad7-b7152db8b872-kube-api-access-fw8jp\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:39:03.400388 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.400351 2573 generic.go:356] "Generic (PLEG): container finished" podID="78b542af-05ce-45a0-bad7-b7152db8b872" containerID="7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2" exitCode=0 Jul 20 19:39:03.400388 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.400393 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zzssr" Jul 20 19:39:03.400892 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.400396 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zzssr" event={"ID":"78b542af-05ce-45a0-bad7-b7152db8b872","Type":"ContainerDied","Data":"7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2"} Jul 20 19:39:03.400892 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.400499 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zzssr" event={"ID":"78b542af-05ce-45a0-bad7-b7152db8b872","Type":"ContainerDied","Data":"63ac730f913da01f49d21d1ceb6cfd10273d9d5f731e8841e12185cde33c8192"} Jul 20 19:39:03.400892 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.400529 2573 scope.go:117] "RemoveContainer" containerID="7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2" Jul 20 19:39:03.409705 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.409687 2573 scope.go:117] "RemoveContainer" containerID="7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2" Jul 20 19:39:03.409961 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:39:03.409942 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2\": container with ID starting with 7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2 not found: ID does not exist" containerID="7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2" Jul 20 19:39:03.410003 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.409971 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2"} err="failed to get container status \"7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2\": rpc error: code = NotFound desc = could not find container \"7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2\": container with ID starting with 7e3c7c98c640b0d1d097825e7b01a967a0932b826f447cd342019db8d6b8e2c2 not found: ID does not exist" Jul 20 19:39:03.419536 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.419512 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zzssr"] Jul 20 19:39:03.424860 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:03.424836 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-zzssr"] Jul 20 19:39:04.536698 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:04.536663 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78b542af-05ce-45a0-bad7-b7152db8b872" path="/var/lib/kubelet/pods/78b542af-05ce-45a0-bad7-b7152db8b872/volumes" Jul 20 19:39:34.895917 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.895886 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56f9c4bd45-wbxbm"] Jul 20 19:39:34.896343 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.896297 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78b542af-05ce-45a0-bad7-b7152db8b872" containerName="authorino" Jul 20 19:39:34.896343 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.896309 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="78b542af-05ce-45a0-bad7-b7152db8b872" containerName="authorino" Jul 20 19:39:34.896419 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.896395 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="78b542af-05ce-45a0-bad7-b7152db8b872" containerName="authorino" Jul 20 19:39:34.900697 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.900678 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:34.902908 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.902885 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-7grfb\"" Jul 20 19:39:34.903127 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.903112 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 19:39:34.903403 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.903384 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 19:39:34.913030 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.913002 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56f9c4bd45-wbxbm"] Jul 20 19:39:34.985274 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.985240 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-webhook-cert\") pod \"maas-controller-56f9c4bd45-wbxbm\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:34.985438 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:34.985284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlrks\" (UniqueName: \"kubernetes.io/projected/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-kube-api-access-hlrks\") pod \"maas-controller-56f9c4bd45-wbxbm\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:35.086550 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:35.086516 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-webhook-cert\") pod \"maas-controller-56f9c4bd45-wbxbm\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:35.086737 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:35.086561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlrks\" (UniqueName: \"kubernetes.io/projected/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-kube-api-access-hlrks\") pod \"maas-controller-56f9c4bd45-wbxbm\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:35.089065 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:35.089043 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-webhook-cert\") pod \"maas-controller-56f9c4bd45-wbxbm\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:35.097179 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:35.097151 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlrks\" (UniqueName: \"kubernetes.io/projected/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-kube-api-access-hlrks\") pod \"maas-controller-56f9c4bd45-wbxbm\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:35.211007 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:35.210918 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:35.347001 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:35.346976 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56f9c4bd45-wbxbm"] Jul 20 19:39:35.349535 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:39:35.349507 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287bdee3_0b9a_4e3b_8f93_f954ea9427f4.slice/crio-5e88db12b3847d2e8e5b857188209a20a84d1acaa99175fc9556996d45f002a1 WatchSource:0}: Error finding container 5e88db12b3847d2e8e5b857188209a20a84d1acaa99175fc9556996d45f002a1: Status 404 returned error can't find the container with id 5e88db12b3847d2e8e5b857188209a20a84d1acaa99175fc9556996d45f002a1 Jul 20 19:39:35.554206 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:35.554131 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" event={"ID":"287bdee3-0b9a-4e3b-8f93-f954ea9427f4","Type":"ContainerStarted","Data":"5e88db12b3847d2e8e5b857188209a20a84d1acaa99175fc9556996d45f002a1"} Jul 20 19:39:37.566232 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:37.566199 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" event={"ID":"287bdee3-0b9a-4e3b-8f93-f954ea9427f4","Type":"ContainerStarted","Data":"8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c"} Jul 20 19:39:37.566641 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:37.566307 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:37.585422 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:37.585238 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" podStartSLOduration=1.450690989 podStartE2EDuration="3.585218448s" totalImagesPullingTime="2.134527459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:39:37.585117368 +0000 UTC m=+733.568410323" watchObservedRunningTime="2026-07-20 19:39:37.585218448 +0000 UTC m=+733.568511406" Jul 20 19:39:48.574980 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:48.574949 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:50.095206 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.095170 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56f9c4bd45-wbxbm"] Jul 20 19:39:50.095692 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.095359 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" podUID="287bdee3-0b9a-4e3b-8f93-f954ea9427f4" containerName="manager" containerID="cri-o://8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c" gracePeriod=30 Jul 20 19:39:50.345318 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.345261 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:50.429127 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.429076 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlrks\" (UniqueName: \"kubernetes.io/projected/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-kube-api-access-hlrks\") pod \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " Jul 20 19:39:50.429331 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.429187 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-webhook-cert\") pod \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\" (UID: \"287bdee3-0b9a-4e3b-8f93-f954ea9427f4\") " Jul 20 19:39:50.431356 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.431327 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "287bdee3-0b9a-4e3b-8f93-f954ea9427f4" (UID: "287bdee3-0b9a-4e3b-8f93-f954ea9427f4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 19:39:50.431356 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.431335 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-kube-api-access-hlrks" (OuterVolumeSpecName: "kube-api-access-hlrks") pod "287bdee3-0b9a-4e3b-8f93-f954ea9427f4" (UID: "287bdee3-0b9a-4e3b-8f93-f954ea9427f4"). InnerVolumeSpecName "kube-api-access-hlrks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:39:50.530289 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.530261 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-webhook-cert\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:39:50.530289 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.530288 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlrks\" (UniqueName: \"kubernetes.io/projected/287bdee3-0b9a-4e3b-8f93-f954ea9427f4-kube-api-access-hlrks\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:39:50.640236 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.640197 2573 generic.go:356] "Generic (PLEG): container finished" podID="287bdee3-0b9a-4e3b-8f93-f954ea9427f4" containerID="8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c" exitCode=0 Jul 20 19:39:50.640393 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.640257 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" Jul 20 19:39:50.640393 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.640274 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" event={"ID":"287bdee3-0b9a-4e3b-8f93-f954ea9427f4","Type":"ContainerDied","Data":"8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c"} Jul 20 19:39:50.640393 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.640335 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f9c4bd45-wbxbm" event={"ID":"287bdee3-0b9a-4e3b-8f93-f954ea9427f4","Type":"ContainerDied","Data":"5e88db12b3847d2e8e5b857188209a20a84d1acaa99175fc9556996d45f002a1"} Jul 20 19:39:50.640393 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.640359 2573 scope.go:117] "RemoveContainer" containerID="8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c" Jul 20 19:39:50.649006 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.648978 2573 scope.go:117] "RemoveContainer" containerID="8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c" Jul 20 19:39:50.649276 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:39:50.649254 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c\": container with ID starting with 8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c not found: ID does not exist" containerID="8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c" Jul 20 19:39:50.649378 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.649283 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c"} err="failed to get container status \"8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c\": rpc error: code = NotFound desc = could not find container \"8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c\": container with ID starting with 8b182315917670bea2324ac5c8e4909b794ae618c0a6f6352fbefa8324e94d2c not found: ID does not exist" Jul 20 19:39:50.656967 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.656946 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56f9c4bd45-wbxbm"] Jul 20 19:39:50.661090 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:50.661070 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56f9c4bd45-wbxbm"] Jul 20 19:39:52.537413 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:39:52.537378 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="287bdee3-0b9a-4e3b-8f93-f954ea9427f4" path="/var/lib/kubelet/pods/287bdee3-0b9a-4e3b-8f93-f954ea9427f4/volumes" Jul 20 19:40:14.339855 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.339814 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-47wqf"] Jul 20 19:40:14.340330 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.340236 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="287bdee3-0b9a-4e3b-8f93-f954ea9427f4" containerName="manager" Jul 20 19:40:14.340330 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.340247 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="287bdee3-0b9a-4e3b-8f93-f954ea9427f4" containerName="manager" Jul 20 19:40:14.340330 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.340316 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="287bdee3-0b9a-4e3b-8f93-f954ea9427f4" containerName="manager" Jul 20 19:40:14.348045 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.348022 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.350763 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.350210 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 20 19:40:14.350763 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.350255 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pcrdc\"" Jul 20 19:40:14.351655 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.351628 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-47wqf"] Jul 20 19:40:14.446129 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.446085 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d0199c1-a2a9-4748-a10a-0c6c28c478cd-plugins-config-volume\") pod \"payload-processing-5bf47cd988-47wqf\" (UID: \"0d0199c1-a2a9-4748-a10a-0c6c28c478cd\") " pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.446309 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.446158 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcczk\" (UniqueName: \"kubernetes.io/projected/0d0199c1-a2a9-4748-a10a-0c6c28c478cd-kube-api-access-dcczk\") pod \"payload-processing-5bf47cd988-47wqf\" (UID: \"0d0199c1-a2a9-4748-a10a-0c6c28c478cd\") " pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.547726 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.547684 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d0199c1-a2a9-4748-a10a-0c6c28c478cd-plugins-config-volume\") pod \"payload-processing-5bf47cd988-47wqf\" (UID: \"0d0199c1-a2a9-4748-a10a-0c6c28c478cd\") " pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.547899 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.547795 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcczk\" (UniqueName: \"kubernetes.io/projected/0d0199c1-a2a9-4748-a10a-0c6c28c478cd-kube-api-access-dcczk\") pod \"payload-processing-5bf47cd988-47wqf\" (UID: \"0d0199c1-a2a9-4748-a10a-0c6c28c478cd\") " pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.548439 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.548417 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0d0199c1-a2a9-4748-a10a-0c6c28c478cd-plugins-config-volume\") pod \"payload-processing-5bf47cd988-47wqf\" (UID: \"0d0199c1-a2a9-4748-a10a-0c6c28c478cd\") " pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.556232 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.556203 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcczk\" (UniqueName: \"kubernetes.io/projected/0d0199c1-a2a9-4748-a10a-0c6c28c478cd-kube-api-access-dcczk\") pod \"payload-processing-5bf47cd988-47wqf\" (UID: \"0d0199c1-a2a9-4748-a10a-0c6c28c478cd\") " pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.659953 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.659922 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:14.799076 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:14.799048 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-47wqf"] Jul 20 19:40:14.800966 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:40:14.800942 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0199c1_a2a9_4748_a10a_0c6c28c478cd.slice/crio-4a19fb523314a8b7d6acbac321046f1d21b978f9022302c78dbee879dfde6cb4 WatchSource:0}: Error finding container 4a19fb523314a8b7d6acbac321046f1d21b978f9022302c78dbee879dfde6cb4: Status 404 returned error can't find the container with id 4a19fb523314a8b7d6acbac321046f1d21b978f9022302c78dbee879dfde6cb4 Jul 20 19:40:15.769861 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:15.769824 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" event={"ID":"0d0199c1-a2a9-4748-a10a-0c6c28c478cd","Type":"ContainerStarted","Data":"4a19fb523314a8b7d6acbac321046f1d21b978f9022302c78dbee879dfde6cb4"} Jul 20 19:40:16.776399 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:16.776366 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" event={"ID":"0d0199c1-a2a9-4748-a10a-0c6c28c478cd","Type":"ContainerStarted","Data":"bb831023ff6ceb55f18433d961764828b9460edce394c7d564185d0d3a7d0e43"} Jul 20 19:40:16.776781 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:16.776502 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:40:16.792176 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:16.792119 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" podStartSLOduration=1.340583407 podStartE2EDuration="2.792103609s" totalImagesPullingTime="1.451520202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:40:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:40:16.790563423 +0000 UTC m=+772.773856397" watchObservedRunningTime="2026-07-20 19:40:16.792103609 +0000 UTC m=+772.775396563" Jul 20 19:40:27.783782 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:40:27.783748 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-47wqf" Jul 20 19:41:07.579921 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.579883 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm"] Jul 20 19:41:07.584360 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.584337 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.586576 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.586551 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 20 19:41:07.586576 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.586550 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 19:41:07.587134 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.587112 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bjz7v\"" Jul 20 19:41:07.587240 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.587118 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 19:41:07.592231 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.592210 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm"] Jul 20 19:41:07.695046 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.695014 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.695229 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.695059 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.695229 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.695095 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.695229 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.695124 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.695229 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.695162 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntfx2\" (UniqueName: \"kubernetes.io/projected/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-kube-api-access-ntfx2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.695229 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.695184 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.795763 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.795734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796001 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.795786 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796001 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.795824 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796001 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.795852 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796001 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.795875 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796001 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.795940 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfx2\" (UniqueName: \"kubernetes.io/projected/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-kube-api-access-ntfx2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796265 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.796223 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796324 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.796268 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.796324 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.796298 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.798208 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.798184 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.798347 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.798316 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.804350 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.804326 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntfx2\" (UniqueName: \"kubernetes.io/projected/167c86d6-5fd4-4283-8134-8f5ab90c0c2f-kube-api-access-ntfx2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm\" (UID: \"167c86d6-5fd4-4283-8134-8f5ab90c0c2f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:07.897806 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:07.897775 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:08.033076 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:08.033051 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm"] Jul 20 19:41:08.035165 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:41:08.035141 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167c86d6_5fd4_4283_8134_8f5ab90c0c2f.slice/crio-0cbe4c3b71de4a889a1b59c6dba0b5bc5b8525fba290db6e1a3fbf742fae170a WatchSource:0}: Error finding container 0cbe4c3b71de4a889a1b59c6dba0b5bc5b8525fba290db6e1a3fbf742fae170a: Status 404 returned error can't find the container with id 0cbe4c3b71de4a889a1b59c6dba0b5bc5b8525fba290db6e1a3fbf742fae170a Jul 20 19:41:09.024457 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:09.024414 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" event={"ID":"167c86d6-5fd4-4283-8134-8f5ab90c0c2f","Type":"ContainerStarted","Data":"0cbe4c3b71de4a889a1b59c6dba0b5bc5b8525fba290db6e1a3fbf742fae170a"} Jul 20 19:41:10.031942 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:10.031905 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" event={"ID":"167c86d6-5fd4-4283-8134-8f5ab90c0c2f","Type":"ContainerStarted","Data":"6a3c1959b08d1430a31be73c01497596db4e6374b61934c75e2868196e3a9c68"} Jul 20 19:41:10.049697 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:10.049639 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" podStartSLOduration=1.897157617 podStartE2EDuration="3.049623793s" totalImagesPullingTime="1.152466176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:41:10.047540581 +0000 UTC m=+826.030833537" watchObservedRunningTime="2026-07-20 19:41:10.049623793 +0000 UTC m=+826.032916740" Jul 20 19:41:17.898034 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:17.897990 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:17.898415 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:17.898051 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:17.910487 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:17.910457 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:18.080261 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:18.080228 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm" Jul 20 19:41:21.180313 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.180278 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx"] Jul 20 19:41:21.182752 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.182733 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.184799 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.184770 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 20 19:41:21.192765 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.192744 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx"] Jul 20 19:41:21.318758 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.318709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b8a9a65-a5e3-4063-b254-090986f51b60-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.318939 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.318782 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.318939 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.318811 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.318939 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.318842 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tskn8\" (UniqueName: \"kubernetes.io/projected/4b8a9a65-a5e3-4063-b254-090986f51b60-kube-api-access-tskn8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.318939 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.318908 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.319084 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.318963 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419367 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419330 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419545 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419380 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419545 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419406 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b8a9a65-a5e3-4063-b254-090986f51b60-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419545 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419450 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419545 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419487 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419545 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419530 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tskn8\" (UniqueName: \"kubernetes.io/projected/4b8a9a65-a5e3-4063-b254-090986f51b60-kube-api-access-tskn8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419881 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419852 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419996 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419905 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.419996 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.419934 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.421980 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.421957 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b8a9a65-a5e3-4063-b254-090986f51b60-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.422128 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.422111 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b8a9a65-a5e3-4063-b254-090986f51b60-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.427860 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.427839 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tskn8\" (UniqueName: \"kubernetes.io/projected/4b8a9a65-a5e3-4063-b254-090986f51b60-kube-api-access-tskn8\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-46pmx\" (UID: \"4b8a9a65-a5e3-4063-b254-090986f51b60\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.493811 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.493735 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:21.626163 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:21.626135 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx"] Jul 20 19:41:21.629291 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:41:21.629264 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b8a9a65_a5e3_4063_b254_090986f51b60.slice/crio-71299c25a7c241c8be4b6b7f732aff4016b64dbf3575f9513e89ad44537830f2 WatchSource:0}: Error finding container 71299c25a7c241c8be4b6b7f732aff4016b64dbf3575f9513e89ad44537830f2: Status 404 returned error can't find the container with id 71299c25a7c241c8be4b6b7f732aff4016b64dbf3575f9513e89ad44537830f2 Jul 20 19:41:22.090977 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:22.090947 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" event={"ID":"4b8a9a65-a5e3-4063-b254-090986f51b60","Type":"ContainerStarted","Data":"3ecaea142efef7aa04ec146fe752355712d994b7b92ea0a93d004c227f8c7583"} Jul 20 19:41:22.091102 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:22.091010 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" event={"ID":"4b8a9a65-a5e3-4063-b254-090986f51b60","Type":"ContainerStarted","Data":"71299c25a7c241c8be4b6b7f732aff4016b64dbf3575f9513e89ad44537830f2"} Jul 20 19:41:22.112929 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:22.112883 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" podStartSLOduration=0.726703356 podStartE2EDuration="1.112868025s" totalImagesPullingTime="386.164669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:41:22.110697179 +0000 UTC m=+838.093990155" watchObservedRunningTime="2026-07-20 19:41:22.112868025 +0000 UTC m=+838.096160979" Jul 20 19:41:31.494757 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:31.494713 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:31.495159 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:31.494827 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:31.507507 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:31.507475 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:41:32.151457 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:41:32.151420 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-46pmx" Jul 20 19:42:24.516889 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:42:24.516813 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:42:24.518762 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:42:24.518741 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:43:06.304580 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.304542 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7dcc46b87f-57dzm"] Jul 20 19:43:06.308159 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.308142 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.310843 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.310819 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 20 19:43:06.310946 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.310819 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2qtj\"" Jul 20 19:43:06.316059 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.316039 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dcc46b87f-57dzm"] Jul 20 19:43:06.376776 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.376741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5bhh\" (UniqueName: \"kubernetes.io/projected/0ca47f48-a1a4-4361-86cd-dc996e24fd3e-kube-api-access-c5bhh\") pod \"authorino-7dcc46b87f-57dzm\" (UID: \"0ca47f48-a1a4-4361-86cd-dc996e24fd3e\") " pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.376932 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.376797 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ca47f48-a1a4-4361-86cd-dc996e24fd3e-tls-cert\") pod \"authorino-7dcc46b87f-57dzm\" (UID: \"0ca47f48-a1a4-4361-86cd-dc996e24fd3e\") " pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.478328 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.478294 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bhh\" (UniqueName: \"kubernetes.io/projected/0ca47f48-a1a4-4361-86cd-dc996e24fd3e-kube-api-access-c5bhh\") pod \"authorino-7dcc46b87f-57dzm\" (UID: \"0ca47f48-a1a4-4361-86cd-dc996e24fd3e\") " pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.478496 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.478355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ca47f48-a1a4-4361-86cd-dc996e24fd3e-tls-cert\") pod \"authorino-7dcc46b87f-57dzm\" (UID: \"0ca47f48-a1a4-4361-86cd-dc996e24fd3e\") " pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.480884 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.480860 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ca47f48-a1a4-4361-86cd-dc996e24fd3e-tls-cert\") pod \"authorino-7dcc46b87f-57dzm\" (UID: \"0ca47f48-a1a4-4361-86cd-dc996e24fd3e\") " pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.486779 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.486746 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5bhh\" (UniqueName: \"kubernetes.io/projected/0ca47f48-a1a4-4361-86cd-dc996e24fd3e-kube-api-access-c5bhh\") pod \"authorino-7dcc46b87f-57dzm\" (UID: \"0ca47f48-a1a4-4361-86cd-dc996e24fd3e\") " pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.618773 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.618748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7dcc46b87f-57dzm" Jul 20 19:43:06.751527 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.751500 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7dcc46b87f-57dzm"] Jul 20 19:43:06.753667 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:43:06.753639 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0ca47f48_a1a4_4361_86cd_dc996e24fd3e.slice/crio-d9744249f82c8f7a280d1fa41e99cdd359b145ca929f0d488511c1385f4f0be7 WatchSource:0}: Error finding container d9744249f82c8f7a280d1fa41e99cdd359b145ca929f0d488511c1385f4f0be7: Status 404 returned error can't find the container with id d9744249f82c8f7a280d1fa41e99cdd359b145ca929f0d488511c1385f4f0be7 Jul 20 19:43:06.754977 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:06.754957 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 19:43:07.611830 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:07.611762 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dcc46b87f-57dzm" event={"ID":"0ca47f48-a1a4-4361-86cd-dc996e24fd3e","Type":"ContainerStarted","Data":"d007e5e0170b8bbc8b5752da69e45d21c4612aa56d3eb2f9a785fea7cdfd3448"} Jul 20 19:43:07.611830 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:07.611809 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7dcc46b87f-57dzm" event={"ID":"0ca47f48-a1a4-4361-86cd-dc996e24fd3e","Type":"ContainerStarted","Data":"d9744249f82c8f7a280d1fa41e99cdd359b145ca929f0d488511c1385f4f0be7"} Jul 20 19:43:07.628262 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:43:07.628099 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7dcc46b87f-57dzm" podStartSLOduration=0.92503299 podStartE2EDuration="1.628077258s" totalImagesPullingTime="703.044268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:43:07.626278922 +0000 UTC m=+943.609571878" watchObservedRunningTime="2026-07-20 19:43:07.628077258 +0000 UTC m=+943.611370214" Jul 20 19:47:24.548464 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:47:24.548436 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:47:24.552007 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:47:24.551984 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:50:10.463743 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:10.463694 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5"] Jul 20 19:50:10.464247 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:10.463907 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" podUID="e70446cf-bcca-4823-9662-60025b2a14a3" containerName="manager" containerID="cri-o://83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1" gracePeriod=10 Jul 20 19:50:14.321539 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.321510 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:50:14.438533 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.438498 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6bw9\" (UniqueName: \"kubernetes.io/projected/e70446cf-bcca-4823-9662-60025b2a14a3-kube-api-access-h6bw9\") pod \"e70446cf-bcca-4823-9662-60025b2a14a3\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " Jul 20 19:50:14.438754 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.438578 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70446cf-bcca-4823-9662-60025b2a14a3-extensions-socket-volume\") pod \"e70446cf-bcca-4823-9662-60025b2a14a3\" (UID: \"e70446cf-bcca-4823-9662-60025b2a14a3\") " Jul 20 19:50:14.438947 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.438922 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e70446cf-bcca-4823-9662-60025b2a14a3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e70446cf-bcca-4823-9662-60025b2a14a3" (UID: "e70446cf-bcca-4823-9662-60025b2a14a3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 19:50:14.440588 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.440564 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e70446cf-bcca-4823-9662-60025b2a14a3-kube-api-access-h6bw9" (OuterVolumeSpecName: "kube-api-access-h6bw9") pod "e70446cf-bcca-4823-9662-60025b2a14a3" (UID: "e70446cf-bcca-4823-9662-60025b2a14a3"). InnerVolumeSpecName "kube-api-access-h6bw9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 19:50:14.539725 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.539693 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6bw9\" (UniqueName: \"kubernetes.io/projected/e70446cf-bcca-4823-9662-60025b2a14a3-kube-api-access-h6bw9\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:50:14.539725 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.539719 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e70446cf-bcca-4823-9662-60025b2a14a3-extensions-socket-volume\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 19:50:14.781832 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.781747 2573 generic.go:356] "Generic (PLEG): container finished" podID="e70446cf-bcca-4823-9662-60025b2a14a3" containerID="83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1" exitCode=0 Jul 20 19:50:14.781832 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.781788 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" event={"ID":"e70446cf-bcca-4823-9662-60025b2a14a3","Type":"ContainerDied","Data":"83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1"} Jul 20 19:50:14.781832 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.781793 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" Jul 20 19:50:14.781832 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.781830 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5" event={"ID":"e70446cf-bcca-4823-9662-60025b2a14a3","Type":"ContainerDied","Data":"64a9f5a1b4c5f3e2f815d17a59dc6830de3ebf5a80be27cb329100e4fe2005c7"} Jul 20 19:50:14.782114 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.781853 2573 scope.go:117] "RemoveContainer" containerID="83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1" Jul 20 19:50:14.791706 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.791685 2573 scope.go:117] "RemoveContainer" containerID="83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1" Jul 20 19:50:14.791963 ip-10-0-128-93 kubenswrapper[2573]: E0720 19:50:14.791944 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1\": container with ID starting with 83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1 not found: ID does not exist" containerID="83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1" Jul 20 19:50:14.792027 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.791972 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1"} err="failed to get container status \"83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1\": rpc error: code = NotFound desc = could not find container \"83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1\": container with ID starting with 83993b21acdd4f79fd9aa6226beae4b55cae4785897ae457b404299812b6bce1 not found: ID does not exist" Jul 20 19:50:14.799598 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.799576 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5"] Jul 20 19:50:14.804370 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:14.804341 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7p6d5"] Jul 20 19:50:16.537134 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:50:16.537103 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e70446cf-bcca-4823-9662-60025b2a14a3" path="/var/lib/kubelet/pods/e70446cf-bcca-4823-9662-60025b2a14a3/volumes" Jul 20 19:51:11.718298 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.718264 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv"] Jul 20 19:51:11.718790 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.718683 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e70446cf-bcca-4823-9662-60025b2a14a3" containerName="manager" Jul 20 19:51:11.718790 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.718696 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e70446cf-bcca-4823-9662-60025b2a14a3" containerName="manager" Jul 20 19:51:11.718790 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.718775 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e70446cf-bcca-4823-9662-60025b2a14a3" containerName="manager" Jul 20 19:51:11.721941 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.721923 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:11.724365 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.724345 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-7g7tv\"" Jul 20 19:51:11.734485 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.734461 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv"] Jul 20 19:51:11.744036 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.744009 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt5q6\" (UniqueName: \"kubernetes.io/projected/33dc374b-f4df-4a57-a30a-7fd063b4b7de-kube-api-access-gt5q6\") pod \"kuadrant-operator-controller-manager-86878c769f-s9nbv\" (UID: \"33dc374b-f4df-4a57-a30a-7fd063b4b7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:11.744141 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.744062 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33dc374b-f4df-4a57-a30a-7fd063b4b7de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s9nbv\" (UID: \"33dc374b-f4df-4a57-a30a-7fd063b4b7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:11.845324 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.845289 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt5q6\" (UniqueName: \"kubernetes.io/projected/33dc374b-f4df-4a57-a30a-7fd063b4b7de-kube-api-access-gt5q6\") pod \"kuadrant-operator-controller-manager-86878c769f-s9nbv\" (UID: \"33dc374b-f4df-4a57-a30a-7fd063b4b7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:11.845511 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.845334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33dc374b-f4df-4a57-a30a-7fd063b4b7de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s9nbv\" (UID: \"33dc374b-f4df-4a57-a30a-7fd063b4b7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:11.845722 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.845704 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/33dc374b-f4df-4a57-a30a-7fd063b4b7de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-s9nbv\" (UID: \"33dc374b-f4df-4a57-a30a-7fd063b4b7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:11.853460 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:11.853434 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt5q6\" (UniqueName: \"kubernetes.io/projected/33dc374b-f4df-4a57-a30a-7fd063b4b7de-kube-api-access-gt5q6\") pod \"kuadrant-operator-controller-manager-86878c769f-s9nbv\" (UID: \"33dc374b-f4df-4a57-a30a-7fd063b4b7de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:12.032467 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:12.032365 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:12.172252 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:12.172226 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv"] Jul 20 19:51:12.175153 ip-10-0-128-93 kubenswrapper[2573]: W0720 19:51:12.175127 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33dc374b_f4df_4a57_a30a_7fd063b4b7de.slice/crio-82fc5b7cf721f19578874f45676624b476829be10856a7f5f01ea04c5fb407cd WatchSource:0}: Error finding container 82fc5b7cf721f19578874f45676624b476829be10856a7f5f01ea04c5fb407cd: Status 404 returned error can't find the container with id 82fc5b7cf721f19578874f45676624b476829be10856a7f5f01ea04c5fb407cd Jul 20 19:51:13.064735 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:13.064682 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" event={"ID":"33dc374b-f4df-4a57-a30a-7fd063b4b7de","Type":"ContainerStarted","Data":"d5cc6ae0731d836a790e4448096d650e381a7f2b6f1b565279acf2341938f463"} Jul 20 19:51:13.065139 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:13.064752 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" event={"ID":"33dc374b-f4df-4a57-a30a-7fd063b4b7de","Type":"ContainerStarted","Data":"82fc5b7cf721f19578874f45676624b476829be10856a7f5f01ea04c5fb407cd"} Jul 20 19:51:13.065139 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:13.064880 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:51:13.092577 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:13.092516 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" podStartSLOduration=2.092498114 podStartE2EDuration="2.092498114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 19:51:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 19:51:13.090450758 +0000 UTC m=+1429.073743714" watchObservedRunningTime="2026-07-20 19:51:13.092498114 +0000 UTC m=+1429.075791069" Jul 20 19:51:24.072060 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:51:24.071980 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-s9nbv" Jul 20 19:52:24.578557 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:52:24.578526 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:52:24.585039 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:52:24.585008 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:57:24.616451 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:57:24.616421 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 19:57:24.626098 ip-10-0-128-93 kubenswrapper[2573]: I0720 19:57:24.626072 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:02:24.654293 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:02:24.654257 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:02:24.663055 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:02:24.663024 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:07:24.687981 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:07:24.687948 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:07:24.696958 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:07:24.696939 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:08:27.440096 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.440060 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9"] Jul 20 20:08:27.443514 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.443498 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.445535 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.445509 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-850607\"/\"default-dockercfg-4p24r\"" Jul 20 20:08:27.445672 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.445551 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-850607\"/\"kube-root-ca.crt\"" Jul 20 20:08:27.446121 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.446102 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-850607\"/\"openshift-service-ca.crt\"" Jul 20 20:08:27.446643 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.446608 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-850607\"/\"test-model-850607-kserve-self-signed-certs\"" Jul 20 20:08:27.458231 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.458210 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9"] Jul 20 20:08:27.511018 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.510978 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tls-certs\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.511218 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.511031 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-dshm\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.511218 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.511049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tmp-dir\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.511218 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.511120 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tqgg\" (UniqueName: \"kubernetes.io/projected/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-kube-api-access-8tqgg\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.511218 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.511166 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-model-cache\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.511379 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.511250 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-home\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.612591 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.612554 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tls-certs\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.612800 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.612636 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-dshm\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.612800 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.612656 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tmp-dir\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.612800 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.612679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tqgg\" (UniqueName: \"kubernetes.io/projected/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-kube-api-access-8tqgg\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.612800 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.612703 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-model-cache\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.612800 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.612768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-home\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.613165 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.613136 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tmp-dir\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.613247 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.613176 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-model-cache\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.613247 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.613207 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-home\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.615387 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.615364 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-dshm\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.615785 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.615766 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tls-certs\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.620871 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.620852 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tqgg\" (UniqueName: \"kubernetes.io/projected/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-kube-api-access-8tqgg\") pod \"test-model-850607-kserve-5f697b46c7-mtvt9\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.753183 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.753099 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:27.910025 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:27.909995 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9"] Jul 20 20:08:27.911675 ip-10-0-128-93 kubenswrapper[2573]: W0720 20:08:27.911645 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bfa14b8_4a52_4d96_b1fc_1852089dbf91.slice/crio-fc75506bcfb8bdc91a5c93daa9e4d89e9b9d4b3165bd3c3550e56b174e3fc503 WatchSource:0}: Error finding container fc75506bcfb8bdc91a5c93daa9e4d89e9b9d4b3165bd3c3550e56b174e3fc503: Status 404 returned error can't find the container with id fc75506bcfb8bdc91a5c93daa9e4d89e9b9d4b3165bd3c3550e56b174e3fc503 Jul 20 20:08:28.225870 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:28.225831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" event={"ID":"0bfa14b8-4a52-4d96-b1fc-1852089dbf91","Type":"ContainerStarted","Data":"f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac"} Jul 20 20:08:28.226044 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:28.225881 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" event={"ID":"0bfa14b8-4a52-4d96-b1fc-1852089dbf91","Type":"ContainerStarted","Data":"fc75506bcfb8bdc91a5c93daa9e4d89e9b9d4b3165bd3c3550e56b174e3fc503"} Jul 20 20:08:28.246380 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:28.246328 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" podStartSLOduration=1.246312371 podStartE2EDuration="1.246312371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:08:28.244218901 +0000 UTC m=+2464.227511857" watchObservedRunningTime="2026-07-20 20:08:28.246312371 +0000 UTC m=+2464.229605324" Jul 20 20:08:37.753991 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:37.753955 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:37.754513 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:37.754231 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:37.767020 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:37.766992 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:08:38.297645 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:08:38.297602 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:09:14.670707 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:14.670672 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9"] Jul 20 20:09:29.691132 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:29.691025 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" podUID="0bfa14b8-4a52-4d96-b1fc-1852089dbf91" containerName="main" containerID="cri-o://f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac" gracePeriod=45 Jul 20 20:09:29.945062 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:29.945005 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:09:30.089655 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.089601 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tmp-dir\") pod \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " Jul 20 20:09:30.089840 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.089706 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tls-certs\") pod \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " Jul 20 20:09:30.089840 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.089756 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-dshm\") pod \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " Jul 20 20:09:30.089840 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.089781 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-home\") pod \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " Jul 20 20:09:30.089840 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.089810 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tqgg\" (UniqueName: \"kubernetes.io/projected/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-kube-api-access-8tqgg\") pod \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " Jul 20 20:09:30.090042 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.089841 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-model-cache\") pod \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\" (UID: \"0bfa14b8-4a52-4d96-b1fc-1852089dbf91\") " Jul 20 20:09:30.090042 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.089907 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0bfa14b8-4a52-4d96-b1fc-1852089dbf91" (UID: "0bfa14b8-4a52-4d96-b1fc-1852089dbf91"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:30.090150 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.090032 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-home" (OuterVolumeSpecName: "home") pod "0bfa14b8-4a52-4d96-b1fc-1852089dbf91" (UID: "0bfa14b8-4a52-4d96-b1fc-1852089dbf91"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:30.090202 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.090165 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-model-cache" (OuterVolumeSpecName: "model-cache") pod "0bfa14b8-4a52-4d96-b1fc-1852089dbf91" (UID: "0bfa14b8-4a52-4d96-b1fc-1852089dbf91"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:30.090252 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.090201 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tmp-dir\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:09:30.090252 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.090219 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-home\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:09:30.092040 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.092001 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-dshm" (OuterVolumeSpecName: "dshm") pod "0bfa14b8-4a52-4d96-b1fc-1852089dbf91" (UID: "0bfa14b8-4a52-4d96-b1fc-1852089dbf91"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:09:30.092168 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.092097 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-kube-api-access-8tqgg" (OuterVolumeSpecName: "kube-api-access-8tqgg") pod "0bfa14b8-4a52-4d96-b1fc-1852089dbf91" (UID: "0bfa14b8-4a52-4d96-b1fc-1852089dbf91"). InnerVolumeSpecName "kube-api-access-8tqgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:09:30.092168 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.092109 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0bfa14b8-4a52-4d96-b1fc-1852089dbf91" (UID: "0bfa14b8-4a52-4d96-b1fc-1852089dbf91"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 20:09:30.191342 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.191307 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-dshm\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:09:30.191342 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.191342 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tqgg\" (UniqueName: \"kubernetes.io/projected/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-kube-api-access-8tqgg\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:09:30.191561 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.191361 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-model-cache\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:09:30.191561 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.191376 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0bfa14b8-4a52-4d96-b1fc-1852089dbf91-tls-certs\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:09:30.552192 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.552160 2573 generic.go:356] "Generic (PLEG): container finished" podID="0bfa14b8-4a52-4d96-b1fc-1852089dbf91" containerID="f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac" exitCode=0 Jul 20 20:09:30.552342 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.552204 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" Jul 20 20:09:30.552342 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.552242 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" event={"ID":"0bfa14b8-4a52-4d96-b1fc-1852089dbf91","Type":"ContainerDied","Data":"f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac"} Jul 20 20:09:30.552342 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.552280 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9" event={"ID":"0bfa14b8-4a52-4d96-b1fc-1852089dbf91","Type":"ContainerDied","Data":"fc75506bcfb8bdc91a5c93daa9e4d89e9b9d4b3165bd3c3550e56b174e3fc503"} Jul 20 20:09:30.552342 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.552296 2573 scope.go:117] "RemoveContainer" containerID="f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac" Jul 20 20:09:30.561551 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.561534 2573 scope.go:117] "RemoveContainer" containerID="f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac" Jul 20 20:09:30.561818 ip-10-0-128-93 kubenswrapper[2573]: E0720 20:09:30.561801 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac\": container with ID starting with f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac not found: ID does not exist" containerID="f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac" Jul 20 20:09:30.561869 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.561826 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac"} err="failed to get container status \"f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac\": rpc error: code = NotFound desc = could not find container \"f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac\": container with ID starting with f8f3ef0b8780a4d3231ec1b1e15a9b2503c55ba505e06c17622027a2a36790ac not found: ID does not exist" Jul 20 20:09:30.569002 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.568979 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9"] Jul 20 20:09:30.572957 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:30.572938 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-850607/test-model-850607-kserve-5f697b46c7-mtvt9"] Jul 20 20:09:32.537634 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:09:32.537585 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bfa14b8-4a52-4d96-b1fc-1852089dbf91" path="/var/lib/kubelet/pods/0bfa14b8-4a52-4d96-b1fc-1852089dbf91/volumes" Jul 20 20:12:24.720211 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:24.720132 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:12:24.730286 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:24.730266 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:12:54.278110 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.278078 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4"] Jul 20 20:12:54.278519 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.278478 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bfa14b8-4a52-4d96-b1fc-1852089dbf91" containerName="main" Jul 20 20:12:54.278519 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.278489 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bfa14b8-4a52-4d96-b1fc-1852089dbf91" containerName="main" Jul 20 20:12:54.278592 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.278562 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bfa14b8-4a52-4d96-b1fc-1852089dbf91" containerName="main" Jul 20 20:12:54.281730 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.281713 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.284231 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.284208 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"kube-root-ca.crt\"" Jul 20 20:12:54.285311 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.285293 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"auth-test-model-b02932-kserve-self-signed-certs\"" Jul 20 20:12:54.285422 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.285326 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"openshift-service-ca.crt\"" Jul 20 20:12:54.285422 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.285295 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"default-dockercfg-jfbqt\"" Jul 20 20:12:54.292560 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.292536 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4"] Jul 20 20:12:54.345828 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.345796 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvz79\" (UniqueName: \"kubernetes.io/projected/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-kube-api-access-pvz79\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.346011 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.345838 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-dshm\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.346011 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.345862 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-model-cache\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.346011 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.345916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tmp-dir\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.346144 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.346041 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-home\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.346144 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.346094 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tls-certs\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.446702 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.446666 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tls-certs\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.446878 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.446720 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvz79\" (UniqueName: \"kubernetes.io/projected/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-kube-api-access-pvz79\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.446878 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.446755 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-dshm\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.446878 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.446780 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-model-cache\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.446878 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.446794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tmp-dir\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.446878 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.446872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-home\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.447267 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.447236 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-model-cache\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.447374 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.447269 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tmp-dir\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.447374 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.447302 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-home\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.449077 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.449051 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-dshm\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.449183 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.449172 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tls-certs\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.455682 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.455663 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvz79\" (UniqueName: \"kubernetes.io/projected/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-kube-api-access-pvz79\") pod \"auth-test-model-b02932-kserve-5944df4f79-rn4l4\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.592360 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.592271 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:12:54.737695 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:54.737655 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4"] Jul 20 20:12:54.745635 ip-10-0-128-93 kubenswrapper[2573]: W0720 20:12:54.745588 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b42d8a8_46ca_4ccf_99f2_84f89ffff866.slice/crio-c39c881dbe51421d9a34b52abc80ab53e821c6e0f3afc3cb45c25381e18af7d8 WatchSource:0}: Error finding container c39c881dbe51421d9a34b52abc80ab53e821c6e0f3afc3cb45c25381e18af7d8: Status 404 returned error can't find the container with id c39c881dbe51421d9a34b52abc80ab53e821c6e0f3afc3cb45c25381e18af7d8 Jul 20 20:12:55.578815 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:55.578773 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" event={"ID":"5b42d8a8-46ca-4ccf-99f2-84f89ffff866","Type":"ContainerStarted","Data":"97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d"} Jul 20 20:12:55.578815 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:55.578823 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" event={"ID":"5b42d8a8-46ca-4ccf-99f2-84f89ffff866","Type":"ContainerStarted","Data":"c39c881dbe51421d9a34b52abc80ab53e821c6e0f3afc3cb45c25381e18af7d8"} Jul 20 20:12:55.597407 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:12:55.597363 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" podStartSLOduration=1.597346964 podStartE2EDuration="1.597346964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:12:55.595988805 +0000 UTC m=+2731.579281760" watchObservedRunningTime="2026-07-20 20:12:55.597346964 +0000 UTC m=+2731.580639918" Jul 20 20:13:04.593325 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:13:04.593285 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:13:04.593325 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:13:04.593334 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:13:04.606213 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:13:04.606188 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:13:04.633850 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:13:04.633820 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:14:58.299148 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:14:58.299115 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4"] Jul 20 20:15:13.319239 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.319180 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" podUID="5b42d8a8-46ca-4ccf-99f2-84f89ffff866" containerName="main" containerID="cri-o://97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d" gracePeriod=45 Jul 20 20:15:13.589043 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.589019 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:15:13.615722 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.615694 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-model-cache\") pod \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " Jul 20 20:15:13.615894 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.615752 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tls-certs\") pod \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " Jul 20 20:15:13.615894 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.615796 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-home\") pod \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " Jul 20 20:15:13.615894 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.615814 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tmp-dir\") pod \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " Jul 20 20:15:13.615894 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.615839 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvz79\" (UniqueName: \"kubernetes.io/projected/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-kube-api-access-pvz79\") pod \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " Jul 20 20:15:13.616106 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.615912 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-dshm\") pod \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\" (UID: \"5b42d8a8-46ca-4ccf-99f2-84f89ffff866\") " Jul 20 20:15:13.616106 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.615998 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-model-cache" (OuterVolumeSpecName: "model-cache") pod "5b42d8a8-46ca-4ccf-99f2-84f89ffff866" (UID: "5b42d8a8-46ca-4ccf-99f2-84f89ffff866"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:15:13.616197 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.616099 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-home" (OuterVolumeSpecName: "home") pod "5b42d8a8-46ca-4ccf-99f2-84f89ffff866" (UID: "5b42d8a8-46ca-4ccf-99f2-84f89ffff866"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:15:13.616197 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.616114 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5b42d8a8-46ca-4ccf-99f2-84f89ffff866" (UID: "5b42d8a8-46ca-4ccf-99f2-84f89ffff866"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:15:13.616718 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.616336 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-model-cache\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:15:13.616718 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.616361 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-home\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:15:13.616718 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.616373 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tmp-dir\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:15:13.618341 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.618313 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5b42d8a8-46ca-4ccf-99f2-84f89ffff866" (UID: "5b42d8a8-46ca-4ccf-99f2-84f89ffff866"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 20:15:13.618341 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.618311 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-kube-api-access-pvz79" (OuterVolumeSpecName: "kube-api-access-pvz79") pod "5b42d8a8-46ca-4ccf-99f2-84f89ffff866" (UID: "5b42d8a8-46ca-4ccf-99f2-84f89ffff866"). InnerVolumeSpecName "kube-api-access-pvz79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:15:13.618474 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.618415 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-dshm" (OuterVolumeSpecName: "dshm") pod "5b42d8a8-46ca-4ccf-99f2-84f89ffff866" (UID: "5b42d8a8-46ca-4ccf-99f2-84f89ffff866"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:15:13.717104 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.717061 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-tls-certs\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:15:13.717104 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.717104 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvz79\" (UniqueName: \"kubernetes.io/projected/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-kube-api-access-pvz79\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:15:13.717313 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:13.717118 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b42d8a8-46ca-4ccf-99f2-84f89ffff866-dshm\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:15:14.281482 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.281440 2573 generic.go:356] "Generic (PLEG): container finished" podID="5b42d8a8-46ca-4ccf-99f2-84f89ffff866" containerID="97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d" exitCode=0 Jul 20 20:15:14.281482 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.281478 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" Jul 20 20:15:14.281770 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.281527 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" event={"ID":"5b42d8a8-46ca-4ccf-99f2-84f89ffff866","Type":"ContainerDied","Data":"97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d"} Jul 20 20:15:14.281770 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.281568 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4" event={"ID":"5b42d8a8-46ca-4ccf-99f2-84f89ffff866","Type":"ContainerDied","Data":"c39c881dbe51421d9a34b52abc80ab53e821c6e0f3afc3cb45c25381e18af7d8"} Jul 20 20:15:14.281770 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.281587 2573 scope.go:117] "RemoveContainer" containerID="97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d" Jul 20 20:15:14.290934 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.290915 2573 scope.go:117] "RemoveContainer" containerID="97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d" Jul 20 20:15:14.291191 ip-10-0-128-93 kubenswrapper[2573]: E0720 20:15:14.291174 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d\": container with ID starting with 97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d not found: ID does not exist" containerID="97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d" Jul 20 20:15:14.291231 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.291201 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d"} err="failed to get container status \"97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d\": rpc error: code = NotFound desc = could not find container \"97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d\": container with ID starting with 97b7f3c172cae2fa85effc12bec57a25d9b4ef73fa06819595d588184062985d not found: ID does not exist" Jul 20 20:15:14.302025 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.301999 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4"] Jul 20 20:15:14.304403 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.304379 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/auth-test-model-b02932-kserve-5944df4f79-rn4l4"] Jul 20 20:15:14.537547 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:14.537469 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b42d8a8-46ca-4ccf-99f2-84f89ffff866" path="/var/lib/kubelet/pods/5b42d8a8-46ca-4ccf-99f2-84f89ffff866/volumes" Jul 20 20:15:22.889279 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.889201 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd"] Jul 20 20:15:22.889754 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.889634 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b42d8a8-46ca-4ccf-99f2-84f89ffff866" containerName="main" Jul 20 20:15:22.889754 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.889647 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b42d8a8-46ca-4ccf-99f2-84f89ffff866" containerName="main" Jul 20 20:15:22.889754 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.889734 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b42d8a8-46ca-4ccf-99f2-84f89ffff866" containerName="main" Jul 20 20:15:22.893981 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.893961 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:22.896596 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.896569 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"openshift-service-ca.crt\"" Jul 20 20:15:22.896596 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.896589 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"sub-test-model-b02932-kserve-self-signed-certs\"" Jul 20 20:15:22.896805 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.896579 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"kube-root-ca.crt\"" Jul 20 20:15:22.896805 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.896642 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"default-dockercfg-jfbqt\"" Jul 20 20:15:22.901462 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.901400 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd"] Jul 20 20:15:22.993853 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.993810 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-dshm\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:22.993853 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.993855 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tmp-dir\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:22.994084 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.993888 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tls-certs\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:22.994084 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.993913 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n94pt\" (UniqueName: \"kubernetes.io/projected/373b2065-9743-4e6c-8cdf-a2a8faba2e68-kube-api-access-n94pt\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:22.994084 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.994015 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-home\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:22.994084 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:22.994053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-model-cache\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095198 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095160 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-model-cache\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095379 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095224 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-dshm\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095379 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095244 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tmp-dir\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095379 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095264 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tls-certs\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095379 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095282 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n94pt\" (UniqueName: \"kubernetes.io/projected/373b2065-9743-4e6c-8cdf-a2a8faba2e68-kube-api-access-n94pt\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095379 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-home\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095736 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095714 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-model-cache\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095838 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095754 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tmp-dir\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.095838 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.095801 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-home\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.097446 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.097421 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-dshm\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.097923 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.097904 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tls-certs\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.104930 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.104911 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n94pt\" (UniqueName: \"kubernetes.io/projected/373b2065-9743-4e6c-8cdf-a2a8faba2e68-kube-api-access-n94pt\") pod \"sub-test-model-b02932-kserve-58947f6dcf-7t8zd\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.204970 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.204893 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:23.340105 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:23.340080 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd"] Jul 20 20:15:23.342752 ip-10-0-128-93 kubenswrapper[2573]: W0720 20:15:23.342724 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod373b2065_9743_4e6c_8cdf_a2a8faba2e68.slice/crio-74e728c38305b6d6f4086dc5f20fed1faafa3b68e69cbf2f4fcbcd1096e00e23 WatchSource:0}: Error finding container 74e728c38305b6d6f4086dc5f20fed1faafa3b68e69cbf2f4fcbcd1096e00e23: Status 404 returned error can't find the container with id 74e728c38305b6d6f4086dc5f20fed1faafa3b68e69cbf2f4fcbcd1096e00e23 Jul 20 20:15:24.338665 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:24.338627 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" event={"ID":"373b2065-9743-4e6c-8cdf-a2a8faba2e68","Type":"ContainerStarted","Data":"a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116"} Jul 20 20:15:24.338665 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:24.338668 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" event={"ID":"373b2065-9743-4e6c-8cdf-a2a8faba2e68","Type":"ContainerStarted","Data":"74e728c38305b6d6f4086dc5f20fed1faafa3b68e69cbf2f4fcbcd1096e00e23"} Jul 20 20:15:24.361576 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:24.361517 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" podStartSLOduration=2.361503184 podStartE2EDuration="2.361503184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:15:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:15:24.359911951 +0000 UTC m=+2880.343204905" watchObservedRunningTime="2026-07-20 20:15:24.361503184 +0000 UTC m=+2880.344796140" Jul 20 20:15:33.205960 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:33.205921 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:33.206412 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:33.205980 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:33.218381 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:33.218360 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:15:33.396068 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:15:33.396037 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:16:14.096907 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:14.096872 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd"] Jul 20 20:16:29.123793 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.123729 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" podUID="373b2065-9743-4e6c-8cdf-a2a8faba2e68" containerName="main" containerID="cri-o://a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116" gracePeriod=45 Jul 20 20:16:29.381230 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.381171 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:16:29.509474 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509444 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-home\") pod \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " Jul 20 20:16:29.509474 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509481 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-dshm\") pod \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " Jul 20 20:16:29.509732 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509511 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tmp-dir\") pod \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " Jul 20 20:16:29.509732 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509554 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tls-certs\") pod \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " Jul 20 20:16:29.509732 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509673 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n94pt\" (UniqueName: \"kubernetes.io/projected/373b2065-9743-4e6c-8cdf-a2a8faba2e68-kube-api-access-n94pt\") pod \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " Jul 20 20:16:29.509904 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509777 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-home" (OuterVolumeSpecName: "home") pod "373b2065-9743-4e6c-8cdf-a2a8faba2e68" (UID: "373b2065-9743-4e6c-8cdf-a2a8faba2e68"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:16:29.509904 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509792 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-model-cache\") pod \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\" (UID: \"373b2065-9743-4e6c-8cdf-a2a8faba2e68\") " Jul 20 20:16:29.509904 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509861 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "373b2065-9743-4e6c-8cdf-a2a8faba2e68" (UID: "373b2065-9743-4e6c-8cdf-a2a8faba2e68"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:16:29.510060 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.509981 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-model-cache" (OuterVolumeSpecName: "model-cache") pod "373b2065-9743-4e6c-8cdf-a2a8faba2e68" (UID: "373b2065-9743-4e6c-8cdf-a2a8faba2e68"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:16:29.510187 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.510162 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-model-cache\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:16:29.510261 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.510193 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-home\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:16:29.510261 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.510208 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tmp-dir\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:16:29.511820 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.511794 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/373b2065-9743-4e6c-8cdf-a2a8faba2e68-kube-api-access-n94pt" (OuterVolumeSpecName: "kube-api-access-n94pt") pod "373b2065-9743-4e6c-8cdf-a2a8faba2e68" (UID: "373b2065-9743-4e6c-8cdf-a2a8faba2e68"). InnerVolumeSpecName "kube-api-access-n94pt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:16:29.511929 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.511848 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-dshm" (OuterVolumeSpecName: "dshm") pod "373b2065-9743-4e6c-8cdf-a2a8faba2e68" (UID: "373b2065-9743-4e6c-8cdf-a2a8faba2e68"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:16:29.511929 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.511849 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "373b2065-9743-4e6c-8cdf-a2a8faba2e68" (UID: "373b2065-9743-4e6c-8cdf-a2a8faba2e68"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 20:16:29.611267 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.611237 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373b2065-9743-4e6c-8cdf-a2a8faba2e68-tls-certs\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:16:29.611267 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.611263 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n94pt\" (UniqueName: \"kubernetes.io/projected/373b2065-9743-4e6c-8cdf-a2a8faba2e68-kube-api-access-n94pt\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:16:29.611447 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.611277 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373b2065-9743-4e6c-8cdf-a2a8faba2e68-dshm\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:16:29.669830 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.669743 2573 generic.go:356] "Generic (PLEG): container finished" podID="373b2065-9743-4e6c-8cdf-a2a8faba2e68" containerID="a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116" exitCode=0 Jul 20 20:16:29.669830 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.669780 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" event={"ID":"373b2065-9743-4e6c-8cdf-a2a8faba2e68","Type":"ContainerDied","Data":"a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116"} Jul 20 20:16:29.669830 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.669804 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" Jul 20 20:16:29.669830 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.669830 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd" event={"ID":"373b2065-9743-4e6c-8cdf-a2a8faba2e68","Type":"ContainerDied","Data":"74e728c38305b6d6f4086dc5f20fed1faafa3b68e69cbf2f4fcbcd1096e00e23"} Jul 20 20:16:29.670180 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.669849 2573 scope.go:117] "RemoveContainer" containerID="a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116" Jul 20 20:16:29.679098 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.679077 2573 scope.go:117] "RemoveContainer" containerID="a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116" Jul 20 20:16:29.679374 ip-10-0-128-93 kubenswrapper[2573]: E0720 20:16:29.679355 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116\": container with ID starting with a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116 not found: ID does not exist" containerID="a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116" Jul 20 20:16:29.679469 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.679382 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116"} err="failed to get container status \"a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116\": rpc error: code = NotFound desc = could not find container \"a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116\": container with ID starting with a82c0a6fff37222fbd0f3867192cf83e29810bdad4bf7f9041e8cc386c7d0116 not found: ID does not exist" Jul 20 20:16:29.692090 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.692054 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd"] Jul 20 20:16:29.694237 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:29.694216 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/sub-test-model-b02932-kserve-58947f6dcf-7t8zd"] Jul 20 20:16:30.538583 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:30.538548 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="373b2065-9743-4e6c-8cdf-a2a8faba2e68" path="/var/lib/kubelet/pods/373b2065-9743-4e6c-8cdf-a2a8faba2e68/volumes" Jul 20 20:16:50.294982 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.294889 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf"] Jul 20 20:16:50.297645 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.295449 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="373b2065-9743-4e6c-8cdf-a2a8faba2e68" containerName="main" Jul 20 20:16:50.297645 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.295466 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="373b2065-9743-4e6c-8cdf-a2a8faba2e68" containerName="main" Jul 20 20:16:50.297645 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.295557 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="373b2065-9743-4e6c-8cdf-a2a8faba2e68" containerName="main" Jul 20 20:16:50.298504 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.298485 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.300789 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.300766 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"kube-root-ca.crt\"" Jul 20 20:16:50.300874 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.300766 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"default-dockercfg-jfbqt\"" Jul 20 20:16:50.301519 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.301496 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"openshift-service-ca.crt\"" Jul 20 20:16:50.301605 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.301539 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b02932\"/\"rate-test-model-b02932-kserve-self-signed-certs\"" Jul 20 20:16:50.310199 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.310175 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf"] Jul 20 20:16:50.383972 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.383939 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frk5n\" (UniqueName: \"kubernetes.io/projected/640b87c3-507a-4bab-b2b7-64ed546c3d2a-kube-api-access-frk5n\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.384155 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.383985 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tls-certs\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.384155 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.384007 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tmp-dir\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.384155 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.384085 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-dshm\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.384155 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.384132 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-model-cache\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.384350 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.384175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-home\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.485487 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.485449 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-dshm\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.485487 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.485490 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-model-cache\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.485786 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.485521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-home\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.485786 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.485577 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frk5n\" (UniqueName: \"kubernetes.io/projected/640b87c3-507a-4bab-b2b7-64ed546c3d2a-kube-api-access-frk5n\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.485786 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.485606 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tls-certs\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.485786 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.485658 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tmp-dir\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.486023 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.485998 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-home\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.486130 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.486110 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-model-cache\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.486187 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.486114 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tmp-dir\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.487783 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.487764 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-dshm\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.488178 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.488156 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tls-certs\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.493569 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.493549 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frk5n\" (UniqueName: \"kubernetes.io/projected/640b87c3-507a-4bab-b2b7-64ed546c3d2a-kube-api-access-frk5n\") pod \"rate-test-model-b02932-kserve-7786b8b96c-k6wxf\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.607565 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.607482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:16:50.759451 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.759416 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf"] Jul 20 20:16:50.763271 ip-10-0-128-93 kubenswrapper[2573]: W0720 20:16:50.763240 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod640b87c3_507a_4bab_b2b7_64ed546c3d2a.slice/crio-a439bd23078b0bfb85af06e323ccc5058990769d1c2248ef9736671d55ecc7bf WatchSource:0}: Error finding container a439bd23078b0bfb85af06e323ccc5058990769d1c2248ef9736671d55ecc7bf: Status 404 returned error can't find the container with id a439bd23078b0bfb85af06e323ccc5058990769d1c2248ef9736671d55ecc7bf Jul 20 20:16:50.773192 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:50.773166 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" event={"ID":"640b87c3-507a-4bab-b2b7-64ed546c3d2a","Type":"ContainerStarted","Data":"a439bd23078b0bfb85af06e323ccc5058990769d1c2248ef9736671d55ecc7bf"} Jul 20 20:16:51.779638 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:51.779577 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" event={"ID":"640b87c3-507a-4bab-b2b7-64ed546c3d2a","Type":"ContainerStarted","Data":"c8d2d03c315f02957733dc7eef27db77d5f832eb4a1e972a976dcc727b4dd88f"} Jul 20 20:16:51.799667 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:16:51.799593 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" podStartSLOduration=1.799576322 podStartE2EDuration="1.799576322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 20:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:16:51.797318203 +0000 UTC m=+2967.780611157" watchObservedRunningTime="2026-07-20 20:16:51.799576322 +0000 UTC m=+2967.782869278" Jul 20 20:17:00.607776 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:00.607729 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:17:00.608214 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:00.607793 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:17:00.620396 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:00.620371 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:17:00.833863 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:00.833831 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:17:24.752339 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:24.752312 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:17:24.763090 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:24.763069 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:17:41.870066 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:41.870030 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf"] Jul 20 20:17:56.891948 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:56.891907 2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" podUID="640b87c3-507a-4bab-b2b7-64ed546c3d2a" containerName="main" containerID="cri-o://c8d2d03c315f02957733dc7eef27db77d5f832eb4a1e972a976dcc727b4dd88f" gracePeriod=45 Jul 20 20:17:57.106499 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.106464 2573 generic.go:356] "Generic (PLEG): container finished" podID="640b87c3-507a-4bab-b2b7-64ed546c3d2a" containerID="c8d2d03c315f02957733dc7eef27db77d5f832eb4a1e972a976dcc727b4dd88f" exitCode=0 Jul 20 20:17:57.106666 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.106540 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" event={"ID":"640b87c3-507a-4bab-b2b7-64ed546c3d2a","Type":"ContainerDied","Data":"c8d2d03c315f02957733dc7eef27db77d5f832eb4a1e972a976dcc727b4dd88f"} Jul 20 20:17:57.148758 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.148702 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:17:57.211987 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.211961 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tmp-dir\") pod \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " Jul 20 20:17:57.212168 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212014 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-model-cache\") pod \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " Jul 20 20:17:57.212168 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212039 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tls-certs\") pod \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " Jul 20 20:17:57.212168 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212082 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-home\") pod \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " Jul 20 20:17:57.212168 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212105 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frk5n\" (UniqueName: \"kubernetes.io/projected/640b87c3-507a-4bab-b2b7-64ed546c3d2a-kube-api-access-frk5n\") pod \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " Jul 20 20:17:57.212168 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212139 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-dshm\") pod \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\" (UID: \"640b87c3-507a-4bab-b2b7-64ed546c3d2a\") " Jul 20 20:17:57.212448 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212255 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "640b87c3-507a-4bab-b2b7-64ed546c3d2a" (UID: "640b87c3-507a-4bab-b2b7-64ed546c3d2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:17:57.212448 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212315 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-model-cache" (OuterVolumeSpecName: "model-cache") pod "640b87c3-507a-4bab-b2b7-64ed546c3d2a" (UID: "640b87c3-507a-4bab-b2b7-64ed546c3d2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:17:57.212448 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212336 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-home" (OuterVolumeSpecName: "home") pod "640b87c3-507a-4bab-b2b7-64ed546c3d2a" (UID: "640b87c3-507a-4bab-b2b7-64ed546c3d2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:17:57.212448 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212419 2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-home\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:17:57.212448 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212437 2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tmp-dir\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:17:57.212666 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.212451 2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-model-cache\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:17:57.214318 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.214282 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-dshm" (OuterVolumeSpecName: "dshm") pod "640b87c3-507a-4bab-b2b7-64ed546c3d2a" (UID: "640b87c3-507a-4bab-b2b7-64ed546c3d2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 20:17:57.214437 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.214358 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "640b87c3-507a-4bab-b2b7-64ed546c3d2a" (UID: "640b87c3-507a-4bab-b2b7-64ed546c3d2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 20:17:57.214437 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.214372 2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/640b87c3-507a-4bab-b2b7-64ed546c3d2a-kube-api-access-frk5n" (OuterVolumeSpecName: "kube-api-access-frk5n") pod "640b87c3-507a-4bab-b2b7-64ed546c3d2a" (UID: "640b87c3-507a-4bab-b2b7-64ed546c3d2a"). InnerVolumeSpecName "kube-api-access-frk5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 20:17:57.313045 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.313015 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frk5n\" (UniqueName: \"kubernetes.io/projected/640b87c3-507a-4bab-b2b7-64ed546c3d2a-kube-api-access-frk5n\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:17:57.313045 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.313043 2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/640b87c3-507a-4bab-b2b7-64ed546c3d2a-dshm\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:17:57.313045 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:57.313053 2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/640b87c3-507a-4bab-b2b7-64ed546c3d2a-tls-certs\") on node \"ip-10-0-128-93.ec2.internal\" DevicePath \"\"" Jul 20 20:17:58.113916 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:58.113886 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" Jul 20 20:17:58.114325 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:58.113918 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf" event={"ID":"640b87c3-507a-4bab-b2b7-64ed546c3d2a","Type":"ContainerDied","Data":"a439bd23078b0bfb85af06e323ccc5058990769d1c2248ef9736671d55ecc7bf"} Jul 20 20:17:58.114325 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:58.113969 2573 scope.go:117] "RemoveContainer" containerID="c8d2d03c315f02957733dc7eef27db77d5f832eb4a1e972a976dcc727b4dd88f" Jul 20 20:17:58.134758 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:58.134733 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf"] Jul 20 20:17:58.140534 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:58.140510 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b02932/rate-test-model-b02932-kserve-7786b8b96c-k6wxf"] Jul 20 20:17:58.538572 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:17:58.538494 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="640b87c3-507a-4bab-b2b7-64ed546c3d2a" path="/var/lib/kubelet/pods/640b87c3-507a-4bab-b2b7-64ed546c3d2a/volumes" Jul 20 20:18:45.624437 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:45.624404 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dcc46b87f-57dzm_0ca47f48-a1a4-4361-86cd-dc996e24fd3e/authorino/0.log" Jul 20 20:18:52.223024 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:52.222994 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-595mw_f8a4a6d4-2f96-4fe1-9264-cee91de1b178/copy-manifests/0.log" Jul 20 20:18:52.231015 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:52.230987 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-595mw_f8a4a6d4-2f96-4fe1-9264-cee91de1b178/manager/0.log" Jul 20 20:18:54.799039 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:54.799005 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dcc46b87f-57dzm_0ca47f48-a1a4-4361-86cd-dc996e24fd3e/authorino/0.log" Jul 20 20:18:55.054432 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:55.054351 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2xlxn_247d3f3b-1054-433d-a9b9-fcaecfb6dba5/manager/0.log" Jul 20 20:18:55.171771 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:55.171741 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8ksdr_8af210c1-043f-4d2c-aed6-50395ddb59cd/kuadrant-console-plugin/0.log" Jul 20 20:18:55.429758 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:55.429731 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s9nbv_33dc374b-f4df-4a57-a30a-7fd063b4b7de/manager/0.log" Jul 20 20:18:56.310051 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:56.310012 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d68bcf69-7sfdh_95140f23-e812-4918-b33f-69688ffdf031/kube-auth-proxy/0.log" Jul 20 20:18:56.818991 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:56.818957 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-47wqf_0d0199c1-a2a9-4748-a10a-0c6c28c478cd/payload-processing/0.log" Jul 20 20:18:57.710645 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:57.710602 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzd2sm_167c86d6-5fd4-4283-8134-8f5ab90c0c2f/main/0.log" Jul 20 20:18:57.830909 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:18:57.830882 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-46pmx_4b8a9a65-a5e3-4063-b254-090986f51b60/main/0.log" Jul 20 20:19:06.466663 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:06.466634 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pw8fv_c51a827b-a50d-479f-a6a2-e467ed632aae/global-pull-secret-syncer/0.log" Jul 20 20:19:06.632968 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:06.632936 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vxjjl_e7344064-0d35-4e15-abae-379facb8bff0/konnectivity-agent/0.log" Jul 20 20:19:06.655037 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:06.655007 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-93.ec2.internal_5c31779b826e1e93da8d4838893e9654/haproxy/0.log" Jul 20 20:19:12.110875 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:12.110839 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7dcc46b87f-57dzm_0ca47f48-a1a4-4361-86cd-dc996e24fd3e/authorino/0.log" Jul 20 20:19:12.183531 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:12.183497 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-2xlxn_247d3f3b-1054-433d-a9b9-fcaecfb6dba5/manager/0.log" Jul 20 20:19:12.215993 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:12.215970 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8ksdr_8af210c1-043f-4d2c-aed6-50395ddb59cd/kuadrant-console-plugin/0.log" Jul 20 20:19:12.384931 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:12.384899 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-s9nbv_33dc374b-f4df-4a57-a30a-7fd063b4b7de/manager/0.log" Jul 20 20:19:14.161990 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.161961 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a5762909-8d95-4936-8af8-36b1f0480735/alertmanager/0.log" Jul 20 20:19:14.194884 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.194854 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a5762909-8d95-4936-8af8-36b1f0480735/config-reloader/0.log" Jul 20 20:19:14.219682 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.219643 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a5762909-8d95-4936-8af8-36b1f0480735/kube-rbac-proxy-web/0.log" Jul 20 20:19:14.250371 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.250344 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a5762909-8d95-4936-8af8-36b1f0480735/kube-rbac-proxy/0.log" Jul 20 20:19:14.277817 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.277788 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a5762909-8d95-4936-8af8-36b1f0480735/kube-rbac-proxy-metric/0.log" Jul 20 20:19:14.303591 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.303564 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a5762909-8d95-4936-8af8-36b1f0480735/prom-label-proxy/0.log" Jul 20 20:19:14.327905 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.327878 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_a5762909-8d95-4936-8af8-36b1f0480735/init-config-reloader/0.log" Jul 20 20:19:14.368785 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.368755 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-pbfvm_3f02c2e1-b01b-441b-9254-6e4a9a286d9c/cluster-monitoring-operator/0.log" Jul 20 20:19:14.485428 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.485343 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5657dfbc9-7njq5_910af318-ff0f-4185-8ba2-b1c64df3e05b/metrics-server/0.log" Jul 20 20:19:14.556670 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.556643 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hjqss_3d74cff2-d67a-498a-9c3d-fc196e32b6f1/node-exporter/0.log" Jul 20 20:19:14.578724 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.578701 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hjqss_3d74cff2-d67a-498a-9c3d-fc196e32b6f1/kube-rbac-proxy/0.log" Jul 20 20:19:14.604554 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.604531 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hjqss_3d74cff2-d67a-498a-9c3d-fc196e32b6f1/init-textfile/0.log" Jul 20 20:19:14.819900 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.819820 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vdwhw_f8d8198b-bd86-41fd-9306-0b59259e2d7d/kube-rbac-proxy-main/0.log" Jul 20 20:19:14.842248 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.842227 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vdwhw_f8d8198b-bd86-41fd-9306-0b59259e2d7d/kube-rbac-proxy-self/0.log" Jul 20 20:19:14.867872 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:14.867848 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vdwhw_f8d8198b-bd86-41fd-9306-0b59259e2d7d/openshift-state-metrics/0.log" Jul 20 20:19:15.103681 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.103574 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-j4bpq_35b485a1-3d66-4c4c-8c7d-5644d47ac29f/prometheus-operator/0.log" Jul 20 20:19:15.125804 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.125779 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-j4bpq_35b485a1-3d66-4c4c-8c7d-5644d47ac29f/kube-rbac-proxy/0.log" Jul 20 20:19:15.159630 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.159588 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-fbzc2_c78be445-1716-4837-8071-338258d7afcb/prometheus-operator-admission-webhook/0.log" Jul 20 20:19:15.199331 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.199309 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74c76db8fb-7zppm_2e626207-ab6d-4ccf-ba7f-da31812687e4/telemeter-client/0.log" Jul 20 20:19:15.227953 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.227930 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74c76db8fb-7zppm_2e626207-ab6d-4ccf-ba7f-da31812687e4/reload/0.log" Jul 20 20:19:15.261156 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.261136 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-74c76db8fb-7zppm_2e626207-ab6d-4ccf-ba7f-da31812687e4/kube-rbac-proxy/0.log" Jul 20 20:19:15.296278 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.296250 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66657987f-6ljng_50fa9f60-7daa-44a7-a7ae-5eefb4921267/thanos-query/0.log" Jul 20 20:19:15.319782 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.319761 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66657987f-6ljng_50fa9f60-7daa-44a7-a7ae-5eefb4921267/kube-rbac-proxy-web/0.log" Jul 20 20:19:15.344014 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.343992 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66657987f-6ljng_50fa9f60-7daa-44a7-a7ae-5eefb4921267/kube-rbac-proxy/0.log" Jul 20 20:19:15.367385 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.367362 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66657987f-6ljng_50fa9f60-7daa-44a7-a7ae-5eefb4921267/prom-label-proxy/0.log" Jul 20 20:19:15.390064 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.390045 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66657987f-6ljng_50fa9f60-7daa-44a7-a7ae-5eefb4921267/kube-rbac-proxy-rules/0.log" Jul 20 20:19:15.413164 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:15.413149 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-66657987f-6ljng_50fa9f60-7daa-44a7-a7ae-5eefb4921267/kube-rbac-proxy-metrics/0.log" Jul 20 20:19:16.472955 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.472915 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j"] Jul 20 20:19:16.473470 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.473450 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="640b87c3-507a-4bab-b2b7-64ed546c3d2a" containerName="main" Jul 20 20:19:16.473568 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.473473 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="640b87c3-507a-4bab-b2b7-64ed546c3d2a" containerName="main" Jul 20 20:19:16.473568 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.473545 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="640b87c3-507a-4bab-b2b7-64ed546c3d2a" containerName="main" Jul 20 20:19:16.476638 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.476602 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.478860 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.478835 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mvcn6\"/\"kube-root-ca.crt\"" Jul 20 20:19:16.479568 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.479549 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mvcn6\"/\"openshift-service-ca.crt\"" Jul 20 20:19:16.479640 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.479550 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mvcn6\"/\"default-dockercfg-66lh9\"" Jul 20 20:19:16.486518 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.486499 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j"] Jul 20 20:19:16.629179 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.629145 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-podres\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.629380 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.629184 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cckvt\" (UniqueName: \"kubernetes.io/projected/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-kube-api-access-cckvt\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.629380 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.629250 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-lib-modules\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.629380 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.629300 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-sys\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.629380 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.629326 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-proc\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.716672 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.716641 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-8vvwl_cb64f499-8e30-48b7-8bbd-7312d893df9e/networking-console-plugin/0.log" Jul 20 20:19:16.730323 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730245 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-podres\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730323 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730278 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cckvt\" (UniqueName: \"kubernetes.io/projected/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-kube-api-access-cckvt\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730323 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730321 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-lib-modules\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730548 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-sys\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730548 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730384 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-proc\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730548 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730398 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-podres\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730548 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730458 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-sys\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730548 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730471 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-proc\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.730548 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.730463 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-lib-modules\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.738083 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.738061 2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cckvt\" (UniqueName: \"kubernetes.io/projected/d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f-kube-api-access-cckvt\") pod \"perf-node-gather-daemonset-vrp2j\" (UID: \"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f\") " pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.786765 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.786728 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:16.921719 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:16.921690 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j"] Jul 20 20:19:16.923683 ip-10-0-128-93 kubenswrapper[2573]: W0720 20:19:16.923654 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd86d7108_758b_4e25_9dcc_e3dfbe4d1c3f.slice/crio-b50734ccf886cd0bdd2016341191fa52f50094be8f5b25ab8e55d931ec4504f7 WatchSource:0}: Error finding container b50734ccf886cd0bdd2016341191fa52f50094be8f5b25ab8e55d931ec4504f7: Status 404 returned error can't find the container with id b50734ccf886cd0bdd2016341191fa52f50094be8f5b25ab8e55d931ec4504f7 Jul 20 20:19:17.372701 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:17.372673 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/2.log" Jul 20 20:19:17.376283 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:17.376257 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-pn76z_857f9a2d-27aa-4663-92bb-64406b0da2a8/console-operator/3.log" Jul 20 20:19:17.503932 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:17.503890 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" event={"ID":"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f","Type":"ContainerStarted","Data":"f32d332c270d5ff362888e74576b61ff41e2d0c74bca0642c9c2683165e27de0"} Jul 20 20:19:17.504320 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:17.503950 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" event={"ID":"d86d7108-758b-4e25-9dcc-e3dfbe4d1c3f","Type":"ContainerStarted","Data":"b50734ccf886cd0bdd2016341191fa52f50094be8f5b25ab8e55d931ec4504f7"} Jul 20 20:19:17.504320 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:17.504018 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:17.521893 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:17.521842 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" podStartSLOduration=1.521828235 podStartE2EDuration="1.521828235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 20:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 20:19:17.519727785 +0000 UTC m=+3113.503020765" watchObservedRunningTime="2026-07-20 20:19:17.521828235 +0000 UTC m=+3113.505121189" Jul 20 20:19:17.921751 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:17.921723 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-w6g6f_4b4e041a-e7a2-48d6-8a4d-2b011319fbd3/download-server/0.log" Jul 20 20:19:18.482571 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:18.482538 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-vnzxb_13f07043-3b16-4458-89cc-a407ca185161/volume-data-source-validator/0.log" Jul 20 20:19:19.562853 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:19.562828 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6gzpv_d45325d1-bac5-4328-abc6-3faa098afeb1/dns/0.log" Jul 20 20:19:19.586060 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:19.586040 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6gzpv_d45325d1-bac5-4328-abc6-3faa098afeb1/kube-rbac-proxy/0.log" Jul 20 20:19:19.714353 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:19.714315 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qxwxz_0d00559d-ac72-41c0-ab2d-68879013c63a/dns-node-resolver/0.log" Jul 20 20:19:20.337821 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:20.337794 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ng274_d7dc7bed-e5e1-494e-af3e-3420e9712c1b/node-ca/0.log" Jul 20 20:19:21.463561 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:21.463525 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d68bcf69-7sfdh_95140f23-e812-4918-b33f-69688ffdf031/kube-auth-proxy/0.log" Jul 20 20:19:21.624435 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:21.624401 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-47wqf_0d0199c1-a2a9-4748-a10a-0c6c28c478cd/payload-processing/0.log" Jul 20 20:19:22.363706 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:22.363677 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x2c5n_704c4f1f-46d5-4fc5-80ed-127d316b940b/serve-healthcheck-canary/0.log" Jul 20 20:19:22.998874 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:22.998848 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-czg79_6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615/kube-rbac-proxy/0.log" Jul 20 20:19:23.022453 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:23.022415 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-czg79_6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615/exporter/0.log" Jul 20 20:19:23.047063 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:23.047034 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-czg79_6c2fe0d0-d05a-4a2d-b34d-31a3cea3e615/extractor/0.log" Jul 20 20:19:23.519312 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:23.519286 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mvcn6/perf-node-gather-daemonset-vrp2j" Jul 20 20:19:25.935732 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:25.935703 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-595mw_f8a4a6d4-2f96-4fe1-9264-cee91de1b178/manager/0.log" Jul 20 20:19:25.959750 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:25.959720 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-859c78d9fd-595mw_f8a4a6d4-2f96-4fe1-9264-cee91de1b178/copy-manifests/0.log" Jul 20 20:19:27.352528 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:27.352497 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d68bcf69-7sfdh_95140f23-e812-4918-b33f-69688ffdf031/kube-auth-proxy/0.log" Jul 20 20:19:27.469399 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:27.469361 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65bc99bbc8-9qlhx_88c32b88-41a4-4dc8-b496-de6ea8a379ac/manager/0.log" Jul 20 20:19:27.478798 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:27.478763 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-47wqf_0d0199c1-a2a9-4748-a10a-0c6c28c478cd/payload-processing/0.log" Jul 20 20:19:28.903228 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:28.903198 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86d68bcf69-7sfdh_95140f23-e812-4918-b33f-69688ffdf031/kube-auth-proxy/0.log" Jul 20 20:19:29.019686 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:29.019655 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-47wqf_0d0199c1-a2a9-4748-a10a-0c6c28c478cd/payload-processing/0.log" Jul 20 20:19:32.616528 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:32.616487 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-6dmpb_148bacec-cc15-4a1a-9e7c-a3ff522ce323/migrator/0.log" Jul 20 20:19:32.644025 ip-10-0-128-93 kubenswrapper[2573]: I0720 20:19:32.643993 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-6dmpb_148bacec-cc15-4a1a-9e7c-a3ff522ce323/graceful-termination/0.log"