Jul 09 04:05:13.460873 ip-10-0-130-250 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 04:05:13.460884 ip-10-0-130-250 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 04:05:13.460891 ip-10-0-130-250 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 04:05:13.461114 ip-10-0-130-250 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 04:05:23.565588 ip-10-0-130-250 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 09 04:05:23.565611 ip-10-0-130-250 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c15e6393e58645db8685fa30e4a42dc3 -- Jul 09 04:07:22.650843 ip-10-0-130-250 systemd[1]: Starting Kubernetes Kubelet... Jul 09 04:07:23.007958 ip-10-0-130-250 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:07:23.007958 ip-10-0-130-250 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 04:07:23.007958 ip-10-0-130-250 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:07:23.007958 ip-10-0-130-250 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 04:07:23.007958 ip-10-0-130-250 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 04:07:23.008884 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.008793 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 04:07:23.012206 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012188 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:07:23.012206 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012206 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012210 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012213 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012216 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012218 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012221 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012225 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012227 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012230 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012232 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012235 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012244 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012246 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012249 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012251 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012254 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012256 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012260 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012262 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:07:23.012274 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012265 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012267 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012270 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012272 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012275 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012278 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012281 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012284 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012287 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012289 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012292 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012294 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012296 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012299 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012302 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012306 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012310 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012313 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012315 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012318 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:07:23.012742 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012320 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012322 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012324 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012327 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012329 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012332 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012334 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012337 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012339 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012342 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012344 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012346 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012349 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012352 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012357 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012360 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012362 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012364 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012367 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012370 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:07:23.013214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012372 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012374 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012377 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012379 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012382 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012384 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012387 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012389 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012392 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012394 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012396 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012399 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012401 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012404 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012408 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012411 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012414 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012418 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012423 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:07:23.013720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012426 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012429 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012431 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012434 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012436 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012439 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012441 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012445 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012447 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012450 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012452 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012455 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012457 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012459 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012463 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012466 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012469 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012471 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012475 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:07:23.014168 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.012477 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:07:23.015454 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015440 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:07:23.015454 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015455 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015459 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015462 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015466 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015469 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015472 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015474 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015477 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015480 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015482 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015485 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015487 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015490 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015492 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015495 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015498 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015500 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015503 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015505 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015507 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:07:23.015508 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015510 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015512 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015515 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015533 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015537 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015539 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015542 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015545 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015547 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015550 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015553 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015563 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015567 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015570 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015572 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015575 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015577 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015580 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:07:23.016051 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015582 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015585 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015587 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015590 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015592 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015594 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015597 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015599 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015602 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015604 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015606 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015609 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015611 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015613 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015616 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015618 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015623 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015626 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015629 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:07:23.016491 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015631 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015635 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015638 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015641 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015644 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015646 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015656 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015658 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015661 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015663 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015666 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015668 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015671 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015673 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015675 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015677 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015681 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015683 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015685 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015688 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:07:23.016974 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015690 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015693 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015695 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015698 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015700 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015702 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015705 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015707 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015710 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015714 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015717 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015719 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015723 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015725 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015728 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015730 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015732 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015735 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015738 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015741 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:07:23.017574 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.015743 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016236 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016242 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016245 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016248 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016251 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016254 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016256 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016259 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016261 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016264 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016267 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016269 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016272 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016274 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016277 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016280 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016283 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016285 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016288 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:07:23.018068 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016290 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016293 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016296 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016298 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016301 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016304 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016306 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016309 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016311 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016313 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016316 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016319 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016321 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016324 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016326 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016328 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016331 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016333 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016336 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016338 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:07:23.018569 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016340 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016343 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016346 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016349 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016354 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016357 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016360 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016363 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016366 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016368 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016372 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016375 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016378 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016381 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016384 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016386 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016389 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016392 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016395 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:07:23.019049 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016397 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016400 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016402 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016405 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016407 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016410 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016412 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016415 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016417 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016419 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016422 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016424 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016428 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016430 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016434 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016437 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016439 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016442 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016444 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016447 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:07:23.019507 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016449 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016453 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016456 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016459 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016462 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016464 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016467 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016469 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016472 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016474 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016477 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016479 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016481 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016484 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016487 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016489 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016492 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016494 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016497 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:07:23.019993 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016500 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.016503 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016586 2584 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016594 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016600 2584 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016604 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016608 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016613 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016617 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016622 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016625 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016628 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016632 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016635 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016638 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016641 2584 flags.go:64] FLAG: --cgroup-root="" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016644 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016647 2584 flags.go:64] FLAG: --client-ca-file="" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016649 2584 flags.go:64] FLAG: --cloud-provider="external" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016653 2584 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016656 2584 flags.go:64] FLAG: --cluster-domain="" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016659 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016662 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016665 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 04:07:23.020442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016668 2584 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016681 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016684 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016687 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016690 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016693 2584 flags.go:64] FLAG: --contention-profiling="false" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016696 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016699 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016707 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016710 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016714 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016717 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016720 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016723 2584 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016725 2584 flags.go:64] FLAG: --enable-server="true" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016731 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016738 2584 flags.go:64] FLAG: --event-burst="100" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016741 2584 flags.go:64] FLAG: --event-qps="50" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016744 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016748 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016751 2584 flags.go:64] FLAG: --eviction-hard="" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016755 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016758 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016761 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016764 2584 flags.go:64] FLAG: --eviction-soft="" Jul 09 04:07:23.021064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016767 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016769 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016772 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016775 2584 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016777 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016780 2584 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016783 2584 flags.go:64] FLAG: --feature-gates="" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016786 2584 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016789 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016791 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016794 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016797 2584 flags.go:64] FLAG: --healthz-port="10248" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016800 2584 flags.go:64] FLAG: --help="false" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016803 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016806 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016808 2584 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016812 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016815 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016818 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016821 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016823 2584 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016826 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016830 2584 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 04:07:23.021675 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016833 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016836 2584 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016838 2584 flags.go:64] FLAG: --kube-reserved="" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016842 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016845 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016848 2584 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016850 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016853 2584 flags.go:64] FLAG: --lock-file="" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016856 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016858 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016862 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016867 2584 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016870 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016872 2584 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016875 2584 flags.go:64] FLAG: --logging-format="text" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016878 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016881 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016884 2584 flags.go:64] FLAG: --manifest-url="" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016887 2584 flags.go:64] FLAG: --manifest-url-header="" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016891 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016894 2584 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016898 2584 flags.go:64] FLAG: --max-pods="110" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016901 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016904 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016906 2584 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 04:07:23.022243 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016909 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016912 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016915 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016918 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016925 2584 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016927 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016931 2584 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016934 2584 flags.go:64] FLAG: --pod-cidr="" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016937 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016943 2584 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016948 2584 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016951 2584 flags.go:64] FLAG: --pods-per-core="0" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016954 2584 flags.go:64] FLAG: --port="10250" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016957 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016959 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00cb0ba7eb876f607" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016963 2584 flags.go:64] FLAG: --qos-reserved="" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016965 2584 flags.go:64] FLAG: --read-only-port="10255" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016968 2584 flags.go:64] FLAG: --register-node="true" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016971 2584 flags.go:64] FLAG: --register-with-taints="" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016975 2584 flags.go:64] FLAG: --registry-burst="10" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016978 2584 flags.go:64] FLAG: --registry-qps="5" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016981 2584 flags.go:64] FLAG: --reserved-cpus="" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016983 2584 flags.go:64] FLAG: --reserved-memory="" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016987 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016990 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 04:07:23.022867 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016993 2584 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016996 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.016998 2584 flags.go:64] FLAG: --runonce="false" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017001 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017004 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017007 2584 flags.go:64] FLAG: --seccomp-default="false" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017010 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017013 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017016 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017021 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017025 2584 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017027 2584 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017030 2584 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017033 2584 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017037 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017039 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017043 2584 flags.go:64] FLAG: --system-cgroups="" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017047 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017053 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017056 2584 flags.go:64] FLAG: --tls-cert-file="" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017059 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017065 2584 flags.go:64] FLAG: --tls-min-version="" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017068 2584 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017070 2584 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017074 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 04:07:23.023477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017076 2584 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017080 2584 flags.go:64] FLAG: --v="2" Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017084 2584 flags.go:64] FLAG: --version="false" Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017088 2584 flags.go:64] FLAG: --vmodule="" Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017091 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017095 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017230 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017234 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017237 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017240 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017243 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017246 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017249 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017252 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017255 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017257 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017260 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017268 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017272 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017276 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017278 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:07:23.024094 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017282 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017285 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017288 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017292 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017294 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017297 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017299 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017302 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017304 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017307 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017309 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017311 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017314 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017316 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017318 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017321 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017323 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017325 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017328 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:07:23.024873 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017330 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017332 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017334 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017337 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017339 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017342 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017344 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017347 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017349 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017351 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017361 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017364 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017366 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017370 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017373 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017375 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017379 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017382 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017384 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017387 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:07:23.025548 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017389 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017393 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017396 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017399 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017402 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017404 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017407 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017410 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017412 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017415 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017417 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017420 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017422 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017424 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017427 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017430 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017432 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017435 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017437 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:07:23.026046 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017441 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017444 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017447 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017449 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017457 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017459 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017464 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017466 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017468 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017472 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017475 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017477 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017480 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017482 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017484 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017487 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017489 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017491 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017494 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017496 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:07:23.026509 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017498 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017501 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017503 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017506 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017508 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.017510 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.017516 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.024962 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.024982 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.025004 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.025013 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025170 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025177 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025184 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:07:23.027030 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025190 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025195 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025200 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025204 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025209 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025213 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025223 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025227 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025232 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025236 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025280 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025296 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025300 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025304 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025308 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025311 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025319 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025327 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025332 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025349 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:07:23.027383 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025354 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025359 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025370 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025374 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025380 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025390 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025394 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025398 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025403 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025413 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025418 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025422 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025568 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025573 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025577 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025580 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025584 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025586 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025589 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:07:23.027889 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025592 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025594 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025597 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025600 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025603 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025606 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025608 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025611 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025613 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025616 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025618 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025621 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025623 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025627 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025630 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025632 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025635 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025637 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025640 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025643 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:07:23.028439 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025645 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025648 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025650 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025653 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025655 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025659 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025661 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025664 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025666 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025670 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025673 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025675 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025678 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025681 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025683 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025686 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025688 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025691 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025693 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025696 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:07:23.028987 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025698 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025701 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025703 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025706 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025709 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025711 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025714 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025716 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025719 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025722 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025724 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025726 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025729 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025732 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025734 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025737 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:07:23.029474 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025739 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.025746 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025871 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025878 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025882 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025886 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025888 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025891 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025894 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025897 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025899 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025902 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025905 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025907 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 04:07:23.029947 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025910 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025913 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025915 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025918 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025920 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025923 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025925 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025929 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025932 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025936 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025938 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025941 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025944 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025946 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025949 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025951 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025953 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025956 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025958 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 04:07:23.030304 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025961 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025964 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025967 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025969 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025971 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025974 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025976 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025979 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025981 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025984 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025986 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025989 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025991 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025993 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025996 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.025998 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026000 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026003 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026005 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 04:07:23.030790 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026008 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026010 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026012 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026014 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026017 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026021 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026025 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026028 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026031 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026034 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026036 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026039 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026042 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026044 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026048 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026051 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026053 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026056 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026058 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026061 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 04:07:23.031257 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026064 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026066 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026068 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026071 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026074 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026076 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026090 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026093 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026097 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026100 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026102 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026105 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026107 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026110 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026112 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026115 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026117 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026120 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026123 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026125 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 04:07:23.031759 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026127 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026130 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026133 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026135 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026137 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026140 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026143 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026146 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:23.026149 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.026154 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.026273 2584 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.028203 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.030222 2584 server.go:1019] "Starting client certificate rotation" Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.030323 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 04:07:23.032253 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.030353 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 04:07:23.032661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.032423 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 04:07:23.033767 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.033748 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 04:07:23.044072 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.044049 2584 log.go:25] "Validated CRI v1 runtime API" Jul 09 04:07:23.046023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.046008 2584 log.go:25] "Validated CRI v1 image API" Jul 09 04:07:23.046914 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.046894 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 04:07:23.049287 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.049265 2584 fs.go:135] Filesystem UUIDs: map[6a30022e-e3fb-4a9c-9cdc-9a46e596db77:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 858c45d6-e39a-4e91-b82d-d1ae7e5813c1:/dev/nvme0n1p3] Jul 09 04:07:23.049374 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.049285 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 04:07:23.055301 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.055173 2584 manager.go:216] Machine: {Timestamp:2026-07-09 04:07:23.053331456 +0000 UTC m=+0.290301625 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3076856 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27da52bc7896a904c51b5fdd1a95b3 SystemUUID:ec27da52-bc78-96a9-04c5-1b5fdd1a95b3 BootID:c15e6393-e586-45db-8685-fa30e4a42dc3 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fe:49:9f:44:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fe:49:9f:44:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:ad:96:1c:1f:44 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 04:07:23.055301 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.055291 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 04:07:23.055443 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.055391 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 04:07:23.057906 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.057880 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 04:07:23.058084 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.057908 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-250.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 04:07:23.058365 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.058345 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 04:07:23.058953 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.058938 2584 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 04:07:23.059001 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.058956 2584 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 04:07:23.059001 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.058964 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 04:07:23.059572 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.059560 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 04:07:23.060540 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.060515 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 04:07:23.062943 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.062930 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 09 04:07:23.063183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.063170 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 04:07:23.063390 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.063353 2584 kubelet.go:486] "Attempting to sync node with API server" Jul 09 04:07:23.063471 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.063410 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 04:07:23.063471 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.063437 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 04:07:23.063471 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.063451 2584 kubelet.go:398] "Adding apiserver pod source" Jul 09 04:07:23.063471 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.063466 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 04:07:23.064070 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.064049 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 04:07:23.064154 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.064078 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 04:07:23.066408 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.066384 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 04:07:23.068043 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.068026 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 04:07:23.068115 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.068056 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 04:07:23.068983 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.068971 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 04:07:23.069028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.068988 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 04:07:23.069028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.068995 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 04:07:23.069028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069000 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 04:07:23.069028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069009 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 04:07:23.069028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069014 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 04:07:23.069028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069022 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 04:07:23.069028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069027 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 04:07:23.069232 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069034 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 04:07:23.069232 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069039 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 04:07:23.069232 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069048 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 04:07:23.069232 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069053 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 04:07:23.069737 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069725 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 04:07:23.069737 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.069737 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 04:07:23.073121 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.073101 2584 server.go:1268] "Started kubelet" Jul 09 04:07:23.073420 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.073369 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 04:07:23.073566 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.073441 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 04:07:23.073777 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.073738 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 04:07:23.074268 ip-10-0-130-250 systemd[1]: Started Kubernetes Kubelet. Jul 09 04:07:23.074921 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.074892 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 04:07:23.075033 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.075008 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-250.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 04:07:23.075211 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.075161 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 04:07:23.075211 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.075144 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 04:07:23.076214 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.076197 2584 server.go:310] "Adding debug handlers to kubelet server" Jul 09 04:07:23.078435 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.078406 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 04:07:23.081911 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.081037 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-250.ec2.internal.18c0838121660746 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-250.ec2.internal,UID:ip-10-0-130-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-250.ec2.internal,},FirstTimestamp:2026-07-09 04:07:23.073070918 +0000 UTC m=+0.310041087,LastTimestamp:2026-07-09 04:07:23.073070918 +0000 UTC m=+0.310041087,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-250.ec2.internal,}" Jul 09 04:07:23.082023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.081925 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 04:07:23.082023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.081939 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 04:07:23.082023 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.082014 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.082437 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.082420 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 04:07:23.082760 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.082652 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 04:07:23.082760 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.082686 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 04:07:23.082883 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.082822 2584 reconstruct.go:97] "Volume reconstruction finished" Jul 09 04:07:23.082883 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.082834 2584 reconciler.go:29] "Reconciler: start to sync state" Jul 09 04:07:23.083075 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.082898 2584 factory.go:55] Registering systemd factory Jul 09 04:07:23.083075 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.082912 2584 factory.go:223] Registration of the systemd container factory successfully Jul 09 04:07:23.083256 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.083210 2584 factory.go:153] Registering CRI-O factory Jul 09 04:07:23.083256 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.083225 2584 factory.go:223] Registration of the crio container factory successfully Jul 09 04:07:23.083340 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.083281 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 04:07:23.083340 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.083305 2584 factory.go:103] Registering Raw factory Jul 09 04:07:23.083340 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.083321 2584 manager.go:1196] Started watching for new ooms in manager Jul 09 04:07:23.083780 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.083756 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hq7wp" Jul 09 04:07:23.083910 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.083893 2584 manager.go:349] Starting recovery of all containers Jul 09 04:07:23.085164 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.084238 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 04:07:23.085164 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.084437 2584 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-250.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 04:07:23.094035 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.093824 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hq7wp" Jul 09 04:07:23.098465 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.098444 2584 manager.go:354] Recovery completed Jul 09 04:07:23.103369 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.103353 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:07:23.105150 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.105132 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:07:23.105227 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.105165 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:07:23.105227 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.105177 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:07:23.105691 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.105677 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 04:07:23.105737 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.105691 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:07:23.105737 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.105692 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 04:07:23.105737 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.105728 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 09 04:07:23.106478 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.106463 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:07:23.106561 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.106491 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:07:23.106561 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.106501 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:07:23.106776 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.106764 2584 policy_none.go:49] "None policy: Start" Jul 09 04:07:23.106808 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.106781 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 04:07:23.106808 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.106792 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 04:07:23.107154 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.107060 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-250.ec2.internal.18c08381234f8a30 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-250.ec2.internal,UID:ip-10-0-130-250.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-250.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-250.ec2.internal,},FirstTimestamp:2026-07-09 04:07:23.105151536 +0000 UTC m=+0.342121704,LastTimestamp:2026-07-09 04:07:23.105151536 +0000 UTC m=+0.342121704,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-250.ec2.internal,}" Jul 09 04:07:23.107749 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.107737 2584 policy_none.go:47] "Start" Jul 09 04:07:23.146226 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.146204 2584 manager.go:337] "Starting Device Plugin manager" Jul 09 04:07:23.146393 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.146247 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 04:07:23.146393 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.146260 2584 server.go:87] "Starting device plugin registration server" Jul 09 04:07:23.146610 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.146596 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 04:07:23.146660 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.146614 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 04:07:23.146762 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.146742 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 04:07:23.146857 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.146843 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 04:07:23.146927 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.146857 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 04:07:23.147419 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.147399 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 04:07:23.147502 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.147456 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.222945 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.222904 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 04:07:23.224388 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.224366 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 04:07:23.224486 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.224455 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 04:07:23.224573 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.224501 2584 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 04:07:23.224573 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.224562 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 04:07:23.226509 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.226484 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:07:23.247048 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.247010 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:07:23.248115 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.248095 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:07:23.248234 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.248129 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:07:23.248234 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.248140 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:07:23.248234 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.248167 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.256153 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.256130 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.256285 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.256159 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-250.ec2.internal\": node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.269935 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.269859 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.325448 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.325395 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal"] Jul 09 04:07:23.325666 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.325503 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:07:23.327001 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.326978 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:07:23.327131 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.327018 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:07:23.327131 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.327033 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:07:23.328197 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.328180 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:07:23.328361 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.328344 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.328414 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.328377 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:07:23.329038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.329021 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:07:23.329137 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.329048 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:07:23.329137 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.329061 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:07:23.329137 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.329090 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:07:23.329137 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.329112 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:07:23.329137 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.329121 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:07:23.330268 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.330252 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.330347 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.330284 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 04:07:23.331957 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.331365 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientMemory" Jul 09 04:07:23.331957 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.331393 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 04:07:23.331957 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.331408 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeHasSufficientPID" Jul 09 04:07:23.355503 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.355470 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-250.ec2.internal\" not found" node="ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.360621 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.360600 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-250.ec2.internal\" not found" node="ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.370748 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.370713 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.384995 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.384950 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3643505d187b4b95f72c43849a5465c1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"3643505d187b4b95f72c43849a5465c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.385140 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.385004 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f6014e45b6f02267c2f67b825f59d370-config\") pod \"kube-apiserver-proxy-ip-10-0-130-250.ec2.internal\" (UID: \"f6014e45b6f02267c2f67b825f59d370\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.385140 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.385024 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3643505d187b4b95f72c43849a5465c1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"3643505d187b4b95f72c43849a5465c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.471560 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.471507 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.485940 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.485911 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3643505d187b4b95f72c43849a5465c1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"3643505d187b4b95f72c43849a5465c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.486034 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.485947 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3643505d187b4b95f72c43849a5465c1-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"3643505d187b4b95f72c43849a5465c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.486034 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.485998 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3643505d187b4b95f72c43849a5465c1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"3643505d187b4b95f72c43849a5465c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.486034 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.486019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f6014e45b6f02267c2f67b825f59d370-config\") pod \"kube-apiserver-proxy-ip-10-0-130-250.ec2.internal\" (UID: \"f6014e45b6f02267c2f67b825f59d370\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.486172 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.486044 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f6014e45b6f02267c2f67b825f59d370-config\") pod \"kube-apiserver-proxy-ip-10-0-130-250.ec2.internal\" (UID: \"f6014e45b6f02267c2f67b825f59d370\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.486172 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.486090 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3643505d187b4b95f72c43849a5465c1-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal\" (UID: \"3643505d187b4b95f72c43849a5465c1\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.572440 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.572359 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.657955 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.657906 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.663412 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:23.663394 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Jul 09 04:07:23.672931 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.672909 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.773730 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.773685 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.874307 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.874224 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:23.974864 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:23.974835 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-250.ec2.internal\" not found" Jul 09 04:07:24.023484 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.023449 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:07:24.028620 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.028589 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 04:07:24.028779 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.028759 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 04:07:24.028829 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.028762 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 04:07:24.066493 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.066448 2584 apiserver.go:52] "Watching apiserver" Jul 09 04:07:24.082410 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.082377 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" Jul 09 04:07:24.082410 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.082399 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 04:07:24.087104 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.087076 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 04:07:24.088326 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.088297 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bcknn","openshift-image-registry/node-ca-5wl45","openshift-network-diagnostics/network-check-target-ldjmc","openshift-network-operator/iptables-alerter-r4rhk","openshift-ovn-kubernetes/ovnkube-node-b5qk6","kube-system/konnectivity-agent-xpm7j","openshift-multus/multus-9t6wq","openshift-multus/multus-additional-cni-plugins-wqxhf","openshift-multus/network-metrics-daemon-vqlpv","kube-system/global-pull-secret-syncer-4dwrv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v","openshift-cluster-node-tuning-operator/tuned-gcpjb"] Jul 09 04:07:24.089756 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.089737 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.090954 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.090930 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:24.091070 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.090997 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:24.092174 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.092148 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.092311 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.092250 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.092311 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.092260 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 04:07:24.092311 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.092273 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-vkghk\"" Jul 09 04:07:24.092524 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.092251 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.093231 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.093216 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.094305 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.094275 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.094678 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.094471 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.094893 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.094875 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 04:07:24.095078 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.095060 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 04:07:24.095363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.095347 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:24.095463 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.095438 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:24.095895 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.095849 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 04:07:24.096046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.095916 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 04:07:24.096046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.095933 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.096046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096009 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gfqvq\"" Jul 09 04:07:24.096046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096045 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.096280 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096052 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 04:07:24.096280 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096228 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 04:02:23 +0000 UTC" deadline="2027-12-07 06:57:25.751970182 +0000 UTC" Jul 09 04:07:24.096280 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096255 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12386h50m1.655719756s" Jul 09 04:07:24.096397 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096328 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nc5cw\"" Jul 09 04:07:24.096439 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096392 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.096592 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096567 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 04:07:24.096743 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096706 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.096806 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096738 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 04:07:24.097026 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.096870 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 04:07:24.097126 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.097091 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 04:07:24.097195 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.097162 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6pml2\"" Jul 09 04:07:24.098053 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.098033 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.099302 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.099277 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:07:24.099644 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.099318 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.099644 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.099336 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.099644 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.099337 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.099644 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.099390 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-229w2\"" Jul 09 04:07:24.099644 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.099398 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" Jul 09 04:07:24.100329 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.100290 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.100414 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.100289 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6wzsb\"" Jul 09 04:07:24.100470 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.100452 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.100597 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.100583 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.101833 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.101814 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.101900 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.101862 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.101955 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.101931 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:24.102123 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.102096 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 04:07:24.102280 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.102261 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.102370 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.102353 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qlm5q\"" Jul 09 04:07:24.102969 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.102944 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-66hzb\"" Jul 09 04:07:24.103099 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.103083 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 04:07:24.103176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.103131 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 04:07:24.103474 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.103455 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal"] Jul 09 04:07:24.103554 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.103541 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.105633 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.105615 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 04:07:24.105929 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.105906 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 04:07:24.106031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.105976 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 04:07:24.106624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.106607 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7x4hc\"" Jul 09 04:07:24.108815 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.108550 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 04:07:24.108815 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.108722 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal"] Jul 09 04:07:24.118643 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.118610 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-g5sj9" Jul 09 04:07:24.124990 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.124911 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-g5sj9" Jul 09 04:07:24.183929 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.183902 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 04:07:24.189628 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189594 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-node-log\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.189779 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189633 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovnkube-config\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.189779 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189660 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-cnibin\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.189779 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189683 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-sys-fs\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.189779 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189707 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk2ph\" (UniqueName: \"kubernetes.io/projected/e5450f12-4d48-4380-b616-623f5b43eaa6-kube-api-access-jk2ph\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.189779 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189748 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-var-lib-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189777 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-etc-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189820 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189847 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfq76\" (UniqueName: \"kubernetes.io/projected/abc2f3dd-fab6-4f10-84ff-919540968d94-kube-api-access-vfq76\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189866 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-slash\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189884 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189922 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-cni-multus\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189952 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/432df2bd-0655-4e32-a76b-f21c2840047b-etc-tuned\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.189992 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.190038 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190027 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190056 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-os-release\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190089 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh9p2\" (UniqueName: \"kubernetes.io/projected/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-kube-api-access-lh9p2\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190116 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-sys\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190148 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-var-lib-kubelet\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190179 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b77cdd8d-939d-4119-9384-38c48868d305-serviceca\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190197 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-run-ovn-kubernetes\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190217 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-cni-bin\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190236 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-os-release\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190251 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-log-socket\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190297 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-socket-dir-parent\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190317 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-k8s-cni-cncf-io\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190331 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-conf-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190366 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysconfig\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.190405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190409 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abc2f3dd-fab6-4f10-84ff-919540968d94-iptables-alerter-script\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190435 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4fxv\" (UniqueName: \"kubernetes.io/projected/b77cdd8d-939d-4119-9384-38c48868d305-kube-api-access-k4fxv\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190465 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc76x\" (UniqueName: \"kubernetes.io/projected/d443cd62-48a9-4b4e-9d96-06f3febe5acf-kube-api-access-hc76x\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190492 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-cni-bin\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190518 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-modprobe-d\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190564 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-env-overrides\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190592 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz6mn\" (UniqueName: \"kubernetes.io/projected/457b866c-b844-4825-b6da-19440a29b601-kube-api-access-xz6mn\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190623 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190647 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-kubelet\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190672 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-cni-binary-copy\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190698 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190726 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-socket-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190778 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-multus-certs\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190810 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b77cdd8d-939d-4119-9384-38c48868d305-host\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190833 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-cni-netd\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191054 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190857 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-etc-kubernetes\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190882 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-kubernetes\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190912 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysctl-conf\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190940 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-run\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190956 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5450f12-4d48-4380-b616-623f5b43eaa6-tmp-dir\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190972 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.190996 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d23d0e3-49b5-4448-b5cb-25c430ab0fa5-agent-certs\") pod \"konnectivity-agent-xpm7j\" (UID: \"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5\") " pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191058 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/432df2bd-0655-4e32-a76b-f21c2840047b-tmp\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191088 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5450f12-4d48-4380-b616-623f5b43eaa6-hosts-file\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191108 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-run-netns\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191134 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovn-node-metrics-cert\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191152 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bmph\" (UniqueName: \"kubernetes.io/projected/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-kube-api-access-7bmph\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191169 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-cnibin\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysctl-d\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191200 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-host\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191214 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-ovn\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.191661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191228 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-netns\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191242 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-hostroot\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191261 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-system-cni-dir\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191275 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-systemd\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191299 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-systemd\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191329 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttdbz\" (UniqueName: \"kubernetes.io/projected/432df2bd-0655-4e32-a76b-f21c2840047b-kube-api-access-ttdbz\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191343 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abc2f3dd-fab6-4f10-84ff-919540968d94-host-slash\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191364 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/10a1edc9-d06f-4cf7-b217-9334547578b8-dbus\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191379 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca4651bf-7ec6-4037-b557-882b89f2fb8b-cni-binary-copy\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191394 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-kubelet\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191408 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-systemd-units\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191429 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovnkube-script-lib\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191446 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-registration-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-device-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191476 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-system-cni-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191493 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.192199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191531 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d23d0e3-49b5-4448-b5cb-25c430ab0fa5-konnectivity-ca\") pod \"konnectivity-agent-xpm7j\" (UID: \"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5\") " pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.192808 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191564 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/10a1edc9-d06f-4cf7-b217-9334547578b8-kubelet-config\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.192808 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191584 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-cni-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192808 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191601 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-daemon-config\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192808 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191636 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfl9l\" (UniqueName: \"kubernetes.io/projected/ca4651bf-7ec6-4037-b557-882b89f2fb8b-kube-api-access-mfl9l\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.192808 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.191691 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-lib-modules\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.245496 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.245460 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:07:24.274358 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.274316 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:07:24.292017 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.291975 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc76x\" (UniqueName: \"kubernetes.io/projected/d443cd62-48a9-4b4e-9d96-06f3febe5acf-kube-api-access-hc76x\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.292203 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292028 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-cni-bin\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.292203 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292086 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-cni-bin\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.292323 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292212 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-modprobe-d\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.292323 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292260 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-env-overrides\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.292323 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292293 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz6mn\" (UniqueName: \"kubernetes.io/projected/457b866c-b844-4825-b6da-19440a29b601-kube-api-access-xz6mn\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.292503 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292325 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:24.292503 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292352 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-kubelet\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.292503 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292392 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-modprobe-d\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.292503 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292414 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-kubelet\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.292520 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-cni-binary-copy\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292604 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.292623 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:24.792595571 +0000 UTC m=+2.029565747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292658 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292689 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-socket-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292742 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-multus-certs\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292748 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.292822 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.292820 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.292896 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret podName:10a1edc9-d06f-4cf7-b217-9334547578b8 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:24.792876273 +0000 UTC m=+2.029846436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret") pod "global-pull-secret-syncer-4dwrv" (UID: "10a1edc9-d06f-4cf7-b217-9334547578b8") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292887 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b77cdd8d-939d-4119-9384-38c48868d305-host\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292902 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-multus-certs\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292928 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-socket-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292946 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-cni-netd\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292955 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-env-overrides\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292976 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-etc-kubernetes\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.292931 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b77cdd8d-939d-4119-9384-38c48868d305-host\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293008 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-kubernetes\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293060 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysctl-conf\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293094 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-run\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293061 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-etc-kubernetes\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293064 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-cni-netd\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293120 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5450f12-4d48-4380-b616-623f5b43eaa6-tmp-dir\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293066 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-kubernetes\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293188 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-run\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.293348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293187 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293204 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysctl-conf\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293189 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-cni-binary-copy\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293224 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293226 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d23d0e3-49b5-4448-b5cb-25c430ab0fa5-agent-certs\") pod \"konnectivity-agent-xpm7j\" (UID: \"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5\") " pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293248 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293349 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5450f12-4d48-4380-b616-623f5b43eaa6-tmp-dir\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293364 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/432df2bd-0655-4e32-a76b-f21c2840047b-tmp\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293381 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293400 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5450f12-4d48-4380-b616-623f5b43eaa6-hosts-file\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293419 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-run-netns\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293440 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovn-node-metrics-cert\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293470 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bmph\" (UniqueName: \"kubernetes.io/projected/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-kube-api-access-7bmph\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293495 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-cnibin\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293502 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-run-netns\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysctl-d\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293505 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e5450f12-4d48-4380-b616-623f5b43eaa6-hosts-file\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293563 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 04:07:24.294208 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293631 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-cnibin\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293673 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysctl-d\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293712 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-host\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293745 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-ovn\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293770 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-netns\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293792 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-ovn\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293793 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-hostroot\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293768 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-host\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293814 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-netns\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293830 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-hostroot\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293836 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-system-cni-dir\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293861 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-systemd\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-systemd\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293923 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttdbz\" (UniqueName: \"kubernetes.io/projected/432df2bd-0655-4e32-a76b-f21c2840047b-kube-api-access-ttdbz\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293944 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-system-cni-dir\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293953 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abc2f3dd-fab6-4f10-84ff-919540968d94-host-slash\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293967 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-run-systemd\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.293982 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/10a1edc9-d06f-4cf7-b217-9334547578b8-dbus\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.295230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294011 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-systemd\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294025 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca4651bf-7ec6-4037-b557-882b89f2fb8b-cni-binary-copy\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294053 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-kubelet\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294059 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/abc2f3dd-fab6-4f10-84ff-919540968d94-host-slash\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294082 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-systemd-units\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294109 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovnkube-script-lib\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294133 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-registration-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294159 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-device-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294184 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-system-cni-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294212 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294241 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d23d0e3-49b5-4448-b5cb-25c430ab0fa5-konnectivity-ca\") pod \"konnectivity-agent-xpm7j\" (UID: \"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5\") " pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294270 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/10a1edc9-d06f-4cf7-b217-9334547578b8-kubelet-config\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294295 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-cni-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294318 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-daemon-config\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294344 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfl9l\" (UniqueName: \"kubernetes.io/projected/ca4651bf-7ec6-4037-b557-882b89f2fb8b-kube-api-access-mfl9l\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294369 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-lib-modules\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294395 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-node-log\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.296176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294421 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovnkube-config\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294446 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-cnibin\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294475 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-sys-fs\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294502 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk2ph\" (UniqueName: \"kubernetes.io/projected/e5450f12-4d48-4380-b616-623f5b43eaa6-kube-api-access-jk2ph\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294527 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-var-lib-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-etc-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294582 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294646 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfq76\" (UniqueName: \"kubernetes.io/projected/abc2f3dd-fab6-4f10-84ff-919540968d94-kube-api-access-vfq76\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294659 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ca4651bf-7ec6-4037-b557-882b89f2fb8b-cni-binary-copy\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294674 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-slash\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294739 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/10a1edc9-d06f-4cf7-b217-9334547578b8-dbus\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294715 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294757 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovnkube-script-lib\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-cni-multus\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294799 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/432df2bd-0655-4e32-a76b-f21c2840047b-etc-tuned\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294835 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-etc-selinux\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294853 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294885 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294899 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-registration-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294912 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-os-release\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294932 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-kubelet\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294941 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh9p2\" (UniqueName: \"kubernetes.io/projected/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-kube-api-access-lh9p2\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.294950 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-device-dir\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295001 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-systemd-units\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295008 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295062 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-system-cni-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295094 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-sys\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295122 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-var-lib-kubelet\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295128 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-os-release\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295150 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b77cdd8d-939d-4119-9384-38c48868d305-serviceca\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-run-ovn-kubernetes\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-cni-bin\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295232 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-os-release\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295263 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-log-socket\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.297938 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295291 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-socket-dir-parent\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295318 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-k8s-cni-cncf-io\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295376 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-conf-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295403 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysconfig\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295436 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295448 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abc2f3dd-fab6-4f10-84ff-919540968d94-iptables-alerter-script\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295495 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4fxv\" (UniqueName: \"kubernetes.io/projected/b77cdd8d-939d-4119-9384-38c48868d305-kube-api-access-k4fxv\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295812 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-cnibin\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295878 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-sys-fs\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295935 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-var-lib-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295944 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-cni-bin\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.295978 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-etc-openvswitch\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296090 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-var-lib-kubelet\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296271 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovnkube-config\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296345 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-sys\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296396 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-run-ovn-kubernetes\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296451 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-host-slash\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.298488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296684 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b77cdd8d-939d-4119-9384-38c48868d305-serviceca\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296767 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-socket-dir-parent\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296781 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-lib-modules\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296834 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/457b866c-b844-4825-b6da-19440a29b601-os-release\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296880 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-log-socket\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296925 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-conf-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.296967 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-run-k8s-cni-cncf-io\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297014 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d443cd62-48a9-4b4e-9d96-06f3febe5acf-node-log\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297037 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/10a1edc9-d06f-4cf7-b217-9334547578b8-kubelet-config\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297036 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-daemon-config\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297062 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/432df2bd-0655-4e32-a76b-f21c2840047b-etc-sysconfig\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297046 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d23d0e3-49b5-4448-b5cb-25c430ab0fa5-konnectivity-ca\") pod \"konnectivity-agent-xpm7j\" (UID: \"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5\") " pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297101 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-host-var-lib-cni-multus\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297114 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ca4651bf-7ec6-4037-b557-882b89f2fb8b-multus-cni-dir\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297410 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/abc2f3dd-fab6-4f10-84ff-919540968d94-iptables-alerter-script\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297849 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/432df2bd-0655-4e32-a76b-f21c2840047b-tmp\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297875 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/432df2bd-0655-4e32-a76b-f21c2840047b-etc-tuned\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297951 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d443cd62-48a9-4b4e-9d96-06f3febe5acf-ovn-node-metrics-cert\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.299071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.297984 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/457b866c-b844-4825-b6da-19440a29b601-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.299727 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.298757 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d23d0e3-49b5-4448-b5cb-25c430ab0fa5-agent-certs\") pod \"konnectivity-agent-xpm7j\" (UID: \"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5\") " pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.301141 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.301111 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc76x\" (UniqueName: \"kubernetes.io/projected/d443cd62-48a9-4b4e-9d96-06f3febe5acf-kube-api-access-hc76x\") pod \"ovnkube-node-b5qk6\" (UID: \"d443cd62-48a9-4b4e-9d96-06f3febe5acf\") " pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.301327 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.301309 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz6mn\" (UniqueName: \"kubernetes.io/projected/457b866c-b844-4825-b6da-19440a29b601-kube-api-access-xz6mn\") pod \"multus-additional-cni-plugins-wqxhf\" (UID: \"457b866c-b844-4825-b6da-19440a29b601\") " pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.302361 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.302336 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:24.302484 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.302368 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:24.302484 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.302384 2584 projected.go:196] Error preparing data for projected volume kube-api-access-wwrfc for pod openshift-network-diagnostics/network-check-target-ldjmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:24.302484 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.302455 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc podName:8d7ee2e8-b9cf-4480-9cc7-9922be49c262 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:24.802436709 +0000 UTC m=+2.039406884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wwrfc" (UniqueName: "kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc") pod "network-check-target-ldjmc" (UID: "8d7ee2e8-b9cf-4480-9cc7-9922be49c262") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:24.302744 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.302624 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bmph\" (UniqueName: \"kubernetes.io/projected/d6e4f659-0c56-48fb-a3da-85bcdd1b0982-kube-api-access-7bmph\") pod \"aws-ebs-csi-driver-node-jgf7v\" (UID: \"d6e4f659-0c56-48fb-a3da-85bcdd1b0982\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.303198 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.303168 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttdbz\" (UniqueName: \"kubernetes.io/projected/432df2bd-0655-4e32-a76b-f21c2840047b-kube-api-access-ttdbz\") pod \"tuned-gcpjb\" (UID: \"432df2bd-0655-4e32-a76b-f21c2840047b\") " pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.304028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.304003 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4fxv\" (UniqueName: \"kubernetes.io/projected/b77cdd8d-939d-4119-9384-38c48868d305-kube-api-access-k4fxv\") pod \"node-ca-5wl45\" (UID: \"b77cdd8d-939d-4119-9384-38c48868d305\") " pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.305383 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.305356 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfq76\" (UniqueName: \"kubernetes.io/projected/abc2f3dd-fab6-4f10-84ff-919540968d94-kube-api-access-vfq76\") pod \"iptables-alerter-r4rhk\" (UID: \"abc2f3dd-fab6-4f10-84ff-919540968d94\") " pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.305811 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.305786 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfl9l\" (UniqueName: \"kubernetes.io/projected/ca4651bf-7ec6-4037-b557-882b89f2fb8b-kube-api-access-mfl9l\") pod \"multus-9t6wq\" (UID: \"ca4651bf-7ec6-4037-b557-882b89f2fb8b\") " pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.305969 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.305949 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh9p2\" (UniqueName: \"kubernetes.io/projected/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-kube-api-access-lh9p2\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:24.306033 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.305948 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk2ph\" (UniqueName: \"kubernetes.io/projected/e5450f12-4d48-4380-b616-623f5b43eaa6-kube-api-access-jk2ph\") pod \"node-resolver-bcknn\" (UID: \"e5450f12-4d48-4380-b616-623f5b43eaa6\") " pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.330187 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.330134 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3643505d187b4b95f72c43849a5465c1.slice/crio-23f3ab2c9afdeb4658cef80e54552789aee2ccdc9bd8b9bc84af34b6d08c6cb0 WatchSource:0}: Error finding container 23f3ab2c9afdeb4658cef80e54552789aee2ccdc9bd8b9bc84af34b6d08c6cb0: Status 404 returned error can't find the container with id 23f3ab2c9afdeb4658cef80e54552789aee2ccdc9bd8b9bc84af34b6d08c6cb0 Jul 09 04:07:24.330410 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.330388 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6014e45b6f02267c2f67b825f59d370.slice/crio-1673d643f3791459a21a999424f57ef221ffb0e894a9eb2507c32eba6c1dc0ae WatchSource:0}: Error finding container 1673d643f3791459a21a999424f57ef221ffb0e894a9eb2507c32eba6c1dc0ae: Status 404 returned error can't find the container with id 1673d643f3791459a21a999424f57ef221ffb0e894a9eb2507c32eba6c1dc0ae Jul 09 04:07:24.336928 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.336902 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:07:24.422051 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.421946 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5wl45" Jul 09 04:07:24.429781 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.429747 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:24.431139 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.431109 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77cdd8d_939d_4119_9384_38c48868d305.slice/crio-f5e613b1812d031895679225c27b4be4dbf0e410cd1cf316f7d7580be9e94b3e WatchSource:0}: Error finding container f5e613b1812d031895679225c27b4be4dbf0e410cd1cf316f7d7580be9e94b3e: Status 404 returned error can't find the container with id f5e613b1812d031895679225c27b4be4dbf0e410cd1cf316f7d7580be9e94b3e Jul 09 04:07:24.437803 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.437774 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd443cd62_48a9_4b4e_9d96_06f3febe5acf.slice/crio-e8c45c67af6c225362e6d52cb4d1d274c8326c342ffe9529498f8bd982ec8675 WatchSource:0}: Error finding container e8c45c67af6c225362e6d52cb4d1d274c8326c342ffe9529498f8bd982ec8675: Status 404 returned error can't find the container with id e8c45c67af6c225362e6d52cb4d1d274c8326c342ffe9529498f8bd982ec8675 Jul 09 04:07:24.462006 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.461967 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9t6wq" Jul 09 04:07:24.470225 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.470182 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca4651bf_7ec6_4037_b557_882b89f2fb8b.slice/crio-9dacad4096cf89afc642c4a54974cceb8e918714eb4cc472caeaef304fce7ea3 WatchSource:0}: Error finding container 9dacad4096cf89afc642c4a54974cceb8e918714eb4cc472caeaef304fce7ea3: Status 404 returned error can't find the container with id 9dacad4096cf89afc642c4a54974cceb8e918714eb4cc472caeaef304fce7ea3 Jul 09 04:07:24.477167 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.477140 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" Jul 09 04:07:24.485674 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.485644 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457b866c_b844_4825_b6da_19440a29b601.slice/crio-fe86ca19f96ec84998742595d543601ec84e104e01c5bcf3520be72557a4ab37 WatchSource:0}: Error finding container fe86ca19f96ec84998742595d543601ec84e104e01c5bcf3520be72557a4ab37: Status 404 returned error can't find the container with id fe86ca19f96ec84998742595d543601ec84e104e01c5bcf3520be72557a4ab37 Jul 09 04:07:24.500972 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.500935 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" Jul 09 04:07:24.507289 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.507264 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bcknn" Jul 09 04:07:24.509285 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.509171 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432df2bd_0655_4e32_a76b_f21c2840047b.slice/crio-fb2f6d58a16ef773d31be75318ae915c779f1492abb14ee62e9c651b92b9819c WatchSource:0}: Error finding container fb2f6d58a16ef773d31be75318ae915c779f1492abb14ee62e9c651b92b9819c: Status 404 returned error can't find the container with id fb2f6d58a16ef773d31be75318ae915c779f1492abb14ee62e9c651b92b9819c Jul 09 04:07:24.515512 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.515474 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5450f12_4d48_4380_b616_623f5b43eaa6.slice/crio-a8cf8c3923fa66efeb239df109430fec981a04e6a061cb71c116fb43599ca761 WatchSource:0}: Error finding container a8cf8c3923fa66efeb239df109430fec981a04e6a061cb71c116fb43599ca761: Status 404 returned error can't find the container with id a8cf8c3923fa66efeb239df109430fec981a04e6a061cb71c116fb43599ca761 Jul 09 04:07:24.519508 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.519483 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r4rhk" Jul 09 04:07:24.529937 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.529908 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabc2f3dd_fab6_4f10_84ff_919540968d94.slice/crio-f827fdd91b645b70bbd5eb8fced84d0af06afc643baf4c5e1a84fd6a8dddebd1 WatchSource:0}: Error finding container f827fdd91b645b70bbd5eb8fced84d0af06afc643baf4c5e1a84fd6a8dddebd1: Status 404 returned error can't find the container with id f827fdd91b645b70bbd5eb8fced84d0af06afc643baf4c5e1a84fd6a8dddebd1 Jul 09 04:07:24.541641 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.541598 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:24.547756 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.547729 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" Jul 09 04:07:24.549977 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.549946 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d23d0e3_49b5_4448_b5cb_25c430ab0fa5.slice/crio-5eb73cdf62f0751a217b2dc85836a9f450c01964fdb9ad3d9e2884e5b94039ea WatchSource:0}: Error finding container 5eb73cdf62f0751a217b2dc85836a9f450c01964fdb9ad3d9e2884e5b94039ea: Status 404 returned error can't find the container with id 5eb73cdf62f0751a217b2dc85836a9f450c01964fdb9ad3d9e2884e5b94039ea Jul 09 04:07:24.557444 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:24.557406 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e4f659_0c56_48fb_a3da_85bcdd1b0982.slice/crio-a2ac2f9d374de911a0a75d6703bf961c152d0f66c78ec451d0dfd76d72569338 WatchSource:0}: Error finding container a2ac2f9d374de911a0a75d6703bf961c152d0f66c78ec451d0dfd76d72569338: Status 404 returned error can't find the container with id a2ac2f9d374de911a0a75d6703bf961c152d0f66c78ec451d0dfd76d72569338 Jul 09 04:07:24.798667 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.798563 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:24.798667 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.798608 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:24.798883 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.798699 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:24.798883 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.798704 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:24.798883 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.798762 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:25.798744828 +0000 UTC m=+3.035714986 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:24.798883 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.798777 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret podName:10a1edc9-d06f-4cf7-b217-9334547578b8 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:25.798770401 +0000 UTC m=+3.035740558 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret") pod "global-pull-secret-syncer-4dwrv" (UID: "10a1edc9-d06f-4cf7-b217-9334547578b8") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:24.899016 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.898981 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:24.899207 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.899138 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:24.899207 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.899155 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:24.899207 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.899165 2584 projected.go:196] Error preparing data for projected volume kube-api-access-wwrfc for pod openshift-network-diagnostics/network-check-target-ldjmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:24.899358 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:24.899222 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc podName:8d7ee2e8-b9cf-4480-9cc7-9922be49c262 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:25.899203986 +0000 UTC m=+3.136174149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwrfc" (UniqueName: "kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc") pod "network-check-target-ldjmc" (UID: "8d7ee2e8-b9cf-4480-9cc7-9922be49c262") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:24.943745 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:24.943706 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:07:25.127497 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.127384 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 04:02:24 +0000 UTC" deadline="2028-02-16 01:58:12.229316655 +0000 UTC" Jul 09 04:07:25.127497 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.127429 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14085h50m47.101893476s" Jul 09 04:07:25.227694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.227662 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:25.227877 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.227817 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:25.244390 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.243544 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" event={"ID":"d6e4f659-0c56-48fb-a3da-85bcdd1b0982","Type":"ContainerStarted","Data":"a2ac2f9d374de911a0a75d6703bf961c152d0f66c78ec451d0dfd76d72569338"} Jul 09 04:07:25.261505 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.261457 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bcknn" event={"ID":"e5450f12-4d48-4380-b616-623f5b43eaa6","Type":"ContainerStarted","Data":"a8cf8c3923fa66efeb239df109430fec981a04e6a061cb71c116fb43599ca761"} Jul 09 04:07:25.275396 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.275328 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerStarted","Data":"fe86ca19f96ec84998742595d543601ec84e104e01c5bcf3520be72557a4ab37"} Jul 09 04:07:25.283911 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.283867 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9t6wq" event={"ID":"ca4651bf-7ec6-4037-b557-882b89f2fb8b","Type":"ContainerStarted","Data":"9dacad4096cf89afc642c4a54974cceb8e918714eb4cc472caeaef304fce7ea3"} Jul 09 04:07:25.290873 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.290836 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" event={"ID":"3643505d187b4b95f72c43849a5465c1","Type":"ContainerStarted","Data":"23f3ab2c9afdeb4658cef80e54552789aee2ccdc9bd8b9bc84af34b6d08c6cb0"} Jul 09 04:07:25.297748 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.297713 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xpm7j" event={"ID":"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5","Type":"ContainerStarted","Data":"5eb73cdf62f0751a217b2dc85836a9f450c01964fdb9ad3d9e2884e5b94039ea"} Jul 09 04:07:25.309576 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.309543 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r4rhk" event={"ID":"abc2f3dd-fab6-4f10-84ff-919540968d94","Type":"ContainerStarted","Data":"f827fdd91b645b70bbd5eb8fced84d0af06afc643baf4c5e1a84fd6a8dddebd1"} Jul 09 04:07:25.326411 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.326366 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" event={"ID":"432df2bd-0655-4e32-a76b-f21c2840047b","Type":"ContainerStarted","Data":"fb2f6d58a16ef773d31be75318ae915c779f1492abb14ee62e9c651b92b9819c"} Jul 09 04:07:25.328641 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.328506 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"e8c45c67af6c225362e6d52cb4d1d274c8326c342ffe9529498f8bd982ec8675"} Jul 09 04:07:25.331618 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.331582 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5wl45" event={"ID":"b77cdd8d-939d-4119-9384-38c48868d305","Type":"ContainerStarted","Data":"f5e613b1812d031895679225c27b4be4dbf0e410cd1cf316f7d7580be9e94b3e"} Jul 09 04:07:25.335064 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.335032 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" event={"ID":"f6014e45b6f02267c2f67b825f59d370","Type":"ContainerStarted","Data":"1673d643f3791459a21a999424f57ef221ffb0e894a9eb2507c32eba6c1dc0ae"} Jul 09 04:07:25.798004 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.797967 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 04:07:25.806727 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.806691 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:25.806921 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.806741 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:25.806921 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.806891 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:25.807037 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.806954 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret podName:10a1edc9-d06f-4cf7-b217-9334547578b8 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.806935826 +0000 UTC m=+5.043905986 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret") pod "global-pull-secret-syncer-4dwrv" (UID: "10a1edc9-d06f-4cf7-b217-9334547578b8") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:25.807364 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.807342 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:25.807446 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.807408 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.807390941 +0000 UTC m=+5.044361102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:25.908224 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:25.908179 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:25.908432 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.908357 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:25.908432 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.908382 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:25.908432 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.908396 2584 projected.go:196] Error preparing data for projected volume kube-api-access-wwrfc for pod openshift-network-diagnostics/network-check-target-ldjmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:25.908636 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:25.908460 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc podName:8d7ee2e8-b9cf-4480-9cc7-9922be49c262 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:27.908440227 +0000 UTC m=+5.145410398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwrfc" (UniqueName: "kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc") pod "network-check-target-ldjmc" (UID: "8d7ee2e8-b9cf-4480-9cc7-9922be49c262") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:26.127815 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:26.127688 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 04:02:24 +0000 UTC" deadline="2028-03-08 01:13:56.103773193 +0000 UTC" Jul 09 04:07:26.127815 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:26.127759 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14589h6m29.976019213s" Jul 09 04:07:26.225179 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:26.224908 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:26.225179 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:26.225061 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:26.225179 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:26.225154 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:26.225513 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:26.225279 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:27.225975 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:27.225375 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:27.225975 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.225563 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:27.831171 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:27.831126 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:27.831386 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:27.831185 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:27.831386 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.831362 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:27.831495 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.831430 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret podName:10a1edc9-d06f-4cf7-b217-9334547578b8 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:31.83140892 +0000 UTC m=+9.068379099 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret") pod "global-pull-secret-syncer-4dwrv" (UID: "10a1edc9-d06f-4cf7-b217-9334547578b8") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:27.832794 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.831889 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:27.832794 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.831954 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:31.831935219 +0000 UTC m=+9.068905378 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:27.932906 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:27.932131 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:27.932906 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.932361 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:27.932906 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.932382 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:27.932906 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.932393 2584 projected.go:196] Error preparing data for projected volume kube-api-access-wwrfc for pod openshift-network-diagnostics/network-check-target-ldjmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:27.932906 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:27.932456 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc podName:8d7ee2e8-b9cf-4480-9cc7-9922be49c262 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:31.932434559 +0000 UTC m=+9.169404743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwrfc" (UniqueName: "kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc") pod "network-check-target-ldjmc" (UID: "8d7ee2e8-b9cf-4480-9cc7-9922be49c262") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:28.225698 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:28.224753 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:28.225698 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:28.224917 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:28.225698 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:28.225384 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:28.225698 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:28.225493 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:29.227909 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:29.227864 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:29.228455 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:29.227989 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:30.225415 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:30.225372 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:30.225637 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:30.225580 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:30.226097 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:30.226075 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:30.226188 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:30.226173 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:31.226105 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:31.225903 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:31.226580 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.226209 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:31.872999 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:31.872954 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:31.873167 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:31.873013 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:31.873167 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.873151 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:31.873286 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.873216 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret podName:10a1edc9-d06f-4cf7-b217-9334547578b8 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:39.873197089 +0000 UTC m=+17.110167265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret") pod "global-pull-secret-syncer-4dwrv" (UID: "10a1edc9-d06f-4cf7-b217-9334547578b8") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:31.873636 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.873616 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:31.873724 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.873670 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:39.873652852 +0000 UTC m=+17.110623015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:31.974120 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:31.974070 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:31.974329 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.974311 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:31.974396 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.974337 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:31.974396 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.974352 2584 projected.go:196] Error preparing data for projected volume kube-api-access-wwrfc for pod openshift-network-diagnostics/network-check-target-ldjmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:31.974492 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:31.974415 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc podName:8d7ee2e8-b9cf-4480-9cc7-9922be49c262 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:39.974395313 +0000 UTC m=+17.211365471 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwrfc" (UniqueName: "kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc") pod "network-check-target-ldjmc" (UID: "8d7ee2e8-b9cf-4480-9cc7-9922be49c262") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:32.225486 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:32.225392 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:32.225677 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:32.225557 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:32.225677 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:32.225629 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:32.225792 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:32.225709 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:33.226589 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:33.226553 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:33.227063 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:33.226665 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:34.225443 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:34.225406 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:34.225654 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:34.225574 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:34.226011 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:34.225992 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:34.226101 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:34.226085 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:35.226601 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:35.226563 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:35.227019 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:35.226725 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:36.225601 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:36.225558 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:36.225798 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:36.225558 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:36.225798 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:36.225704 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:36.225798 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:36.225769 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:37.225224 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:37.225138 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:37.225673 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:37.225260 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:38.225127 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:38.225083 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:38.225303 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:38.225146 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:38.225303 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:38.225233 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:38.225740 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:38.225351 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:39.225191 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:39.225151 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:39.225373 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:39.225295 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:39.936232 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:39.936190 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:39.936513 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:39.936239 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:39.936513 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:39.936347 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:39.936513 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:39.936382 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:39.936513 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:39.936429 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:55.936408267 +0000 UTC m=+33.173378442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:39.936513 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:39.936449 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret podName:10a1edc9-d06f-4cf7-b217-9334547578b8 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:55.936439587 +0000 UTC m=+33.173409743 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret") pod "global-pull-secret-syncer-4dwrv" (UID: "10a1edc9-d06f-4cf7-b217-9334547578b8") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:40.036801 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:40.036755 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:40.036978 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:40.036933 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:40.036978 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:40.036963 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:40.036978 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:40.036974 2584 projected.go:196] Error preparing data for projected volume kube-api-access-wwrfc for pod openshift-network-diagnostics/network-check-target-ldjmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:40.037109 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:40.037035 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc podName:8d7ee2e8-b9cf-4480-9cc7-9922be49c262 nodeName:}" failed. No retries permitted until 2026-07-09 04:07:56.037016156 +0000 UTC m=+33.273986332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwrfc" (UniqueName: "kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc") pod "network-check-target-ldjmc" (UID: "8d7ee2e8-b9cf-4480-9cc7-9922be49c262") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:40.225542 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:40.225445 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:40.225972 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:40.225594 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:40.225972 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:40.225649 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:40.225972 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:40.225766 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:41.225658 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:41.225622 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:41.226096 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:41.225760 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:42.225122 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:42.225080 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:42.225122 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:42.225106 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:42.225368 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:42.225227 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:42.225368 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:42.225347 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:43.226037 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:43.226003 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:43.226449 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:43.226133 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:44.225509 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:44.225468 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:44.225720 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:44.225535 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:44.225720 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:44.225608 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:44.225832 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:44.225764 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:45.225550 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:45.225507 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:45.225970 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:45.225635 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:46.224913 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.224732 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:46.225115 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.224780 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:46.225115 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:46.225025 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:46.225115 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:46.225053 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:46.403700 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.403664 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9t6wq" event={"ID":"ca4651bf-7ec6-4037-b557-882b89f2fb8b","Type":"ContainerStarted","Data":"7d096178fb54bf11c39f0c1a6bdcf45c057ebe741d9d58d9768c2d1e19e3c771"} Jul 09 04:07:46.406364 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.406323 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" event={"ID":"432df2bd-0655-4e32-a76b-f21c2840047b","Type":"ContainerStarted","Data":"47006ee06bc543e81a3fc5d250fb2146cc663363a3557952f7337d85deca572a"} Jul 09 04:07:46.409840 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.409816 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"44b870f3509596d64a2059359b64bf3ad00fbf10178208650f98f406eab3c46e"} Jul 09 04:07:46.409945 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.409895 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"316cf01068fa77c83f435a23bf58cf0ba3bd19e785ec7e50fb2b6bd27548196c"} Jul 09 04:07:46.410002 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.409946 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"53871ea22c5de66a719f1f02bc991a93e0e30b80888f4571eaa467c0b95af95c"} Jul 09 04:07:46.410002 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.409992 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"cc36772815dd408a425c0a7963fbbedffeca1b52128bc60713f08edb3fdefb24"} Jul 09 04:07:46.410088 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.410034 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"b2506926f730ce63bab55a2e11232853980c57c0895bfb5175f2f33f2a375782"} Jul 09 04:07:46.410088 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.410062 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"73ea0cd42037fd887e5c287e76a6d182c441919dbf184ddde0604ee11ec28039"} Jul 09 04:07:46.412950 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.412929 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" event={"ID":"f6014e45b6f02267c2f67b825f59d370","Type":"ContainerStarted","Data":"4b0670e9736348fdfa4e86fe9e74ba5fef2dcf379e1f943ce18520e6103e3358"} Jul 09 04:07:46.422633 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.422579 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9t6wq" podStartSLOduration=2.430606916 podStartE2EDuration="23.422563469s" totalImagesPullingTime="20.991956553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:46.422028614 +0000 UTC m=+23.658998793" watchObservedRunningTime="2026-07-09 04:07:46.422563469 +0000 UTC m=+23.659533647" Jul 09 04:07:46.438008 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.437960 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gcpjb" podStartSLOduration=2.486397603 podStartE2EDuration="23.437944659s" totalImagesPullingTime="20.951547056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:46.437448083 +0000 UTC m=+23.674418285" watchObservedRunningTime="2026-07-09 04:07:46.437944659 +0000 UTC m=+23.674914836" Jul 09 04:07:46.449696 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.449624 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-250.ec2.internal" podStartSLOduration=22.449604041 podStartE2EDuration="22.449604041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:46.449604188 +0000 UTC m=+23.686574365" watchObservedRunningTime="2026-07-09 04:07:46.449604041 +0000 UTC m=+23.686574214" Jul 09 04:07:46.464177 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:46.464129 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5wl45" podStartSLOduration=2.570772318 podStartE2EDuration="23.464111721s" totalImagesPullingTime="20.893339403s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:46.463573282 +0000 UTC m=+23.700543461" watchObservedRunningTime="2026-07-09 04:07:46.464111721 +0000 UTC m=+23.701081898" Jul 09 04:07:47.225177 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.225077 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:47.225375 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:47.225213 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:47.416937 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.416656 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" event={"ID":"d6e4f659-0c56-48fb-a3da-85bcdd1b0982","Type":"ContainerStarted","Data":"29822a07a42ea0346d307c6daebae23055ea6b6e95bb51c80ceda724ab5352b8"} Jul 09 04:07:47.418291 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.418257 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bcknn" event={"ID":"e5450f12-4d48-4380-b616-623f5b43eaa6","Type":"ContainerStarted","Data":"340b60d56b69908d991caedfd87ccb8d39baca556867420250f5e0f3f8c6e5f5"} Jul 09 04:07:47.420022 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.419998 2584 generic.go:356] "Generic (PLEG): container finished" podID="457b866c-b844-4825-b6da-19440a29b601" containerID="1aa3f14435039b2e852f2a5112a1e4a3c68db2fc671429658fb87d389b786a89" exitCode=0 Jul 09 04:07:47.420147 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.420082 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerDied","Data":"1aa3f14435039b2e852f2a5112a1e4a3c68db2fc671429658fb87d389b786a89"} Jul 09 04:07:47.422048 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.422022 2584 generic.go:356] "Generic (PLEG): container finished" podID="3643505d187b4b95f72c43849a5465c1" containerID="36082dcb3f275fda33c280fe959fd0f21bf9197e4eab5d9d303905b6a42778f8" exitCode=0 Jul 09 04:07:47.422477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.422454 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" event={"ID":"3643505d187b4b95f72c43849a5465c1","Type":"ContainerDied","Data":"36082dcb3f275fda33c280fe959fd0f21bf9197e4eab5d9d303905b6a42778f8"} Jul 09 04:07:47.424480 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.424454 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xpm7j" event={"ID":"2d23d0e3-49b5-4448-b5cb-25c430ab0fa5","Type":"ContainerStarted","Data":"1d8fa803d1b49bbff692ff4e0f45a1d7c8ba9fc6bd12eab198b8e51b3f59bbdb"} Jul 09 04:07:47.426074 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.426050 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r4rhk" event={"ID":"abc2f3dd-fab6-4f10-84ff-919540968d94","Type":"ContainerStarted","Data":"f54f7db392182be75b0f3a21ccdd97c7e0df26a42bc2818f579b701f63dac520"} Jul 09 04:07:47.428086 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.428066 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5wl45" event={"ID":"b77cdd8d-939d-4119-9384-38c48868d305","Type":"ContainerStarted","Data":"4470010cdc8a5dd5bb31540eaea1799ff01c4e79a0522a504eb10a690698441e"} Jul 09 04:07:47.432635 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.432570 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bcknn" podStartSLOduration=7.285107293 podStartE2EDuration="23.432558553s" totalImagesPullingTime="16.14745126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:47.432354186 +0000 UTC m=+24.669324382" watchObservedRunningTime="2026-07-09 04:07:47.432558553 +0000 UTC m=+24.669528730" Jul 09 04:07:47.475428 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.475281 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xpm7j" podStartSLOduration=7.362606994 podStartE2EDuration="23.475260874s" totalImagesPullingTime="16.11265388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:47.475174121 +0000 UTC m=+24.712144299" watchObservedRunningTime="2026-07-09 04:07:47.475260874 +0000 UTC m=+24.712231053" Jul 09 04:07:47.488451 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:47.488378 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r4rhk" podStartSLOduration=2.558784386 podStartE2EDuration="23.488355051s" totalImagesPullingTime="20.929570665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:47.487840344 +0000 UTC m=+24.724810524" watchObservedRunningTime="2026-07-09 04:07:47.488355051 +0000 UTC m=+24.725325235" Jul 09 04:07:48.045557 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.045514 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 04:07:48.162548 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.162446 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T04:07:48.045550359Z","UUID":"66c29adc-7c08-4ca5-a585-aa2f5cf95c33","Handler":null,"Name":"","Endpoint":""} Jul 09 04:07:48.163880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.163845 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 04:07:48.164023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.163891 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 04:07:48.224980 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.224899 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:48.225152 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.224908 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:48.225152 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:48.225017 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:48.225152 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:48.225111 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:48.433095 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.433062 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"fb4258887f902357f87cc04203ca1226e730a8502cf300ad341cc65952e34b60"} Jul 09 04:07:48.435251 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.435223 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" event={"ID":"d6e4f659-0c56-48fb-a3da-85bcdd1b0982","Type":"ContainerStarted","Data":"3a5a021e391aab11699e30641f6dd902997e2f6158ed4a0863256da7a7c67e07"} Jul 09 04:07:48.440278 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.440243 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" event={"ID":"3643505d187b4b95f72c43849a5465c1","Type":"ContainerStarted","Data":"29f0294f5e4c088c151083e4f06a44decfb2be74fee4f141901f5808efe2c0a8"} Jul 09 04:07:48.455217 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:48.455164 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-250.ec2.internal" podStartSLOduration=24.455150771 podStartE2EDuration="24.455150771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:48.454651121 +0000 UTC m=+25.691621318" watchObservedRunningTime="2026-07-09 04:07:48.455150771 +0000 UTC m=+25.692120948" Jul 09 04:07:49.225279 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:49.225248 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:49.225517 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:49.225375 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:49.444854 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:49.444817 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" event={"ID":"d6e4f659-0c56-48fb-a3da-85bcdd1b0982","Type":"ContainerStarted","Data":"d8f330e195f3685c35f15abbb06299e2ac45b73f4d54c3ea70c52076509a5a90"} Jul 09 04:07:50.225238 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:50.225196 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:50.225433 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:50.225196 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:50.225433 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:50.225315 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:50.225433 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:50.225400 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:51.225492 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.225460 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:51.225979 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:51.225590 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:51.323967 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.323889 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:51.324605 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.324585 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:51.338141 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.338095 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jgf7v" podStartSLOduration=3.947931636 podStartE2EDuration="27.338080041s" totalImagesPullingTime="23.390148405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:49.461939259 +0000 UTC m=+26.698909437" watchObservedRunningTime="2026-07-09 04:07:51.338080041 +0000 UTC m=+28.575050219" Jul 09 04:07:51.461975 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.461877 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" event={"ID":"d443cd62-48a9-4b4e-9d96-06f3febe5acf","Type":"ContainerStarted","Data":"2736ed37229edcb336f865da347e740abe7252fe93fbef5ec07f455228ca1a2a"} Jul 09 04:07:51.462168 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.462125 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:51.462645 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.462619 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xpm7j" Jul 09 04:07:51.493286 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:51.493211 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" podStartSLOduration=7.374615108 podStartE2EDuration="28.493191525s" totalImagesPullingTime="21.118576417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:07:51.492761081 +0000 UTC m=+28.729731259" watchObservedRunningTime="2026-07-09 04:07:51.493191525 +0000 UTC m=+28.730161703" Jul 09 04:07:52.224902 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.224859 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:52.225066 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.224908 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:52.225066 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:52.224995 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:52.225187 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:52.225147 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:52.464855 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.464814 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:52.465421 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.464893 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:52.465421 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.464924 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:52.482319 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.482239 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:52.482750 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.482727 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:07:52.734014 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.733926 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ldjmc"] Jul 09 04:07:52.734151 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.734051 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:52.734209 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:52.734147 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:52.736757 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.736723 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4dwrv"] Jul 09 04:07:52.736883 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.736835 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:52.736951 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:52.736935 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:52.737516 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.737491 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vqlpv"] Jul 09 04:07:52.737677 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:52.737603 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:52.737723 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:52.737691 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:54.225216 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:54.225175 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:54.225850 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:54.225180 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:54.225850 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:54.225318 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:54.225850 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:54.225175 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:54.225850 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:54.225390 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:54.225850 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:54.225480 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:55.472858 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:55.472608 2584 generic.go:356] "Generic (PLEG): container finished" podID="457b866c-b844-4825-b6da-19440a29b601" containerID="f25e0246bc33106aee2da0e8cb73b24e9e3cb7ae54d1c0d28e7be20c76304bc8" exitCode=0 Jul 09 04:07:55.472858 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:55.472701 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerDied","Data":"f25e0246bc33106aee2da0e8cb73b24e9e3cb7ae54d1c0d28e7be20c76304bc8"} Jul 09 04:07:55.951568 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:55.951503 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:55.951734 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:55.951651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:55.951734 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:55.951674 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:55.951842 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:55.951745 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret podName:10a1edc9-d06f-4cf7-b217-9334547578b8 nodeName:}" failed. No retries permitted until 2026-07-09 04:08:27.951729718 +0000 UTC m=+65.188699873 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret") pod "global-pull-secret-syncer-4dwrv" (UID: "10a1edc9-d06f-4cf7-b217-9334547578b8") : object "kube-system"/"original-pull-secret" not registered Jul 09 04:07:55.951842 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:55.951763 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:55.951842 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:55.951805 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:08:27.951792769 +0000 UTC m=+65.188762929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 04:07:56.052218 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:56.052187 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:56.052414 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:56.052381 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 04:07:56.052414 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:56.052405 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 04:07:56.052414 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:56.052415 2584 projected.go:196] Error preparing data for projected volume kube-api-access-wwrfc for pod openshift-network-diagnostics/network-check-target-ldjmc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:56.052583 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:56.052479 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc podName:8d7ee2e8-b9cf-4480-9cc7-9922be49c262 nodeName:}" failed. No retries permitted until 2026-07-09 04:08:28.052460264 +0000 UTC m=+65.289430420 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wwrfc" (UniqueName: "kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc") pod "network-check-target-ldjmc" (UID: "8d7ee2e8-b9cf-4480-9cc7-9922be49c262") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 04:07:56.225091 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:56.225065 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:56.225218 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:56.225100 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:56.225218 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:56.225062 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:56.225218 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:56.225185 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:56.225377 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:56.225294 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:56.225427 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:56.225388 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:56.476784 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:56.476695 2584 generic.go:356] "Generic (PLEG): container finished" podID="457b866c-b844-4825-b6da-19440a29b601" containerID="dd30ab8db84ffc7b4d26cce2d097dfe80157f08c977a55476253d4876f8b3468" exitCode=0 Jul 09 04:07:56.476784 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:56.476738 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerDied","Data":"dd30ab8db84ffc7b4d26cce2d097dfe80157f08c977a55476253d4876f8b3468"} Jul 09 04:07:57.481249 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:57.481210 2584 generic.go:356] "Generic (PLEG): container finished" podID="457b866c-b844-4825-b6da-19440a29b601" containerID="c786c3c744d64d517a32b56661ef7a456e7c4ab53536d031dfde08c9436fa067" exitCode=0 Jul 09 04:07:57.481610 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:57.481272 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerDied","Data":"c786c3c744d64d517a32b56661ef7a456e7c4ab53536d031dfde08c9436fa067"} Jul 09 04:07:58.225024 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:58.224992 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:07:58.225024 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:58.225014 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:07:58.225252 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:58.225085 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:07:58.225252 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:58.225099 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4dwrv" podUID="10a1edc9-d06f-4cf7-b217-9334547578b8" Jul 09 04:07:58.225252 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:58.225163 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:07:58.225252 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:58.225214 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ldjmc" podUID="8d7ee2e8-b9cf-4480-9cc7-9922be49c262" Jul 09 04:07:59.081664 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.081632 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-250.ec2.internal" event="NodeReady" Jul 09 04:07:59.082139 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.081807 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 04:07:59.125300 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.125261 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2q2pz"] Jul 09 04:07:59.131709 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.131673 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-78klw"] Jul 09 04:07:59.131873 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.131811 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.134313 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.134287 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 04:07:59.134557 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.134540 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 04:07:59.134652 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.134556 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbnmf\"" Jul 09 04:07:59.134771 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.134752 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.137144 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.137120 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 04:07:59.137270 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.137161 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 04:07:59.137270 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.137213 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kr28h\"" Jul 09 04:07:59.137444 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.137426 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 04:07:59.137945 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.137852 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2q2pz"] Jul 09 04:07:59.141160 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.141104 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-78klw"] Jul 09 04:07:59.273654 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.273566 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t75l\" (UniqueName: \"kubernetes.io/projected/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-kube-api-access-4t75l\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.273654 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.273630 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.273880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.273667 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-tmp-dir\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.273880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.273692 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwcdv\" (UniqueName: \"kubernetes.io/projected/c3d7be2b-3556-4dda-b2fb-d7ed0c822414-kube-api-access-vwcdv\") pod \"ingress-canary-78klw\" (UID: \"c3d7be2b-3556-4dda-b2fb-d7ed0c822414\") " pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.273880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.273714 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-config-volume\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.273880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.273758 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3d7be2b-3556-4dda-b2fb-d7ed0c822414-cert\") pod \"ingress-canary-78klw\" (UID: \"c3d7be2b-3556-4dda-b2fb-d7ed0c822414\") " pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.374905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.374871 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t75l\" (UniqueName: \"kubernetes.io/projected/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-kube-api-access-4t75l\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.375076 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.374944 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.375076 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.374976 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-tmp-dir\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.375076 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.375003 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwcdv\" (UniqueName: \"kubernetes.io/projected/c3d7be2b-3556-4dda-b2fb-d7ed0c822414-kube-api-access-vwcdv\") pod \"ingress-canary-78klw\" (UID: \"c3d7be2b-3556-4dda-b2fb-d7ed0c822414\") " pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.375076 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.375033 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-config-volume\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.375076 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.375064 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3d7be2b-3556-4dda-b2fb-d7ed0c822414-cert\") pod \"ingress-canary-78klw\" (UID: \"c3d7be2b-3556-4dda-b2fb-d7ed0c822414\") " pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.375322 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:59.375119 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:59.375322 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:59.375188 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:07:59.875165731 +0000 UTC m=+37.112135905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:07:59.375428 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.375370 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-tmp-dir\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.375839 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.375814 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-config-volume\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.382072 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.382041 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c3d7be2b-3556-4dda-b2fb-d7ed0c822414-cert\") pod \"ingress-canary-78klw\" (UID: \"c3d7be2b-3556-4dda-b2fb-d7ed0c822414\") " pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.383999 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.383978 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwcdv\" (UniqueName: \"kubernetes.io/projected/c3d7be2b-3556-4dda-b2fb-d7ed0c822414-kube-api-access-vwcdv\") pod \"ingress-canary-78klw\" (UID: \"c3d7be2b-3556-4dda-b2fb-d7ed0c822414\") " pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.384504 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.384477 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t75l\" (UniqueName: \"kubernetes.io/projected/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-kube-api-access-4t75l\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.453702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.453657 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-78klw" Jul 09 04:07:59.606454 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.606278 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-78klw"] Jul 09 04:07:59.611217 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:07:59.611177 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d7be2b_3556_4dda_b2fb_d7ed0c822414.slice/crio-9d7d83163394b79682eb42080c3e01b34ee258d68fa53c5a363e069ec0b6a481 WatchSource:0}: Error finding container 9d7d83163394b79682eb42080c3e01b34ee258d68fa53c5a363e069ec0b6a481: Status 404 returned error can't find the container with id 9d7d83163394b79682eb42080c3e01b34ee258d68fa53c5a363e069ec0b6a481 Jul 09 04:07:59.879078 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:07:59.878994 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:07:59.879385 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:59.879131 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:07:59.879385 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:07:59.879197 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:08:00.879183476 +0000 UTC m=+38.116153632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:08:00.224850 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.224811 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:08:00.225558 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.224919 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:08:00.225558 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.224927 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:08:00.227640 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.227624 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:08:00.228747 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.228731 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 04:08:00.228830 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.228753 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v2z2w\"" Jul 09 04:08:00.228830 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.228753 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 04:08:00.228935 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.228834 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:08:00.228935 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.228890 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kh5dc\"" Jul 09 04:08:00.491707 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.491619 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-78klw" event={"ID":"c3d7be2b-3556-4dda-b2fb-d7ed0c822414","Type":"ContainerStarted","Data":"9d7d83163394b79682eb42080c3e01b34ee258d68fa53c5a363e069ec0b6a481"} Jul 09 04:08:00.887414 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:00.887378 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:08:00.887634 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:00.887572 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:08:00.887694 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:00.887639 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:08:02.887618584 +0000 UTC m=+40.124588743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:08:02.906047 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:02.906010 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:08:02.906675 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:02.906186 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:08:02.906675 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:02.906267 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:08:06.906246129 +0000 UTC m=+44.143216300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:08:04.504676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:04.504631 2584 generic.go:356] "Generic (PLEG): container finished" podID="457b866c-b844-4825-b6da-19440a29b601" containerID="6f45c2df909d15a51ba0ecfad644ed0df67bfa5caabfd15c11fccf911624e566" exitCode=0 Jul 09 04:08:04.505181 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:04.504709 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerDied","Data":"6f45c2df909d15a51ba0ecfad644ed0df67bfa5caabfd15c11fccf911624e566"} Jul 09 04:08:04.506204 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:04.506174 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-78klw" event={"ID":"c3d7be2b-3556-4dda-b2fb-d7ed0c822414","Type":"ContainerStarted","Data":"84a5491eea5e5abaa90b911e4b8b6437a03b5b522a7200bd79694401ad27aade"} Jul 09 04:08:04.542148 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:04.542078 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-78klw" podStartSLOduration=1.276996815 podStartE2EDuration="5.542054776s" totalImagesPullingTime="4.265057961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:04.540943774 +0000 UTC m=+41.777913955" watchObservedRunningTime="2026-07-09 04:08:04.542054776 +0000 UTC m=+41.779024957" Jul 09 04:08:05.511611 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:05.511577 2584 generic.go:356] "Generic (PLEG): container finished" podID="457b866c-b844-4825-b6da-19440a29b601" containerID="cf04ba0b70a4adb774103db24b1d1752d1a9b9dae619b132ac5f336f70d0b4ca" exitCode=0 Jul 09 04:08:05.512084 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:05.511666 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerDied","Data":"cf04ba0b70a4adb774103db24b1d1752d1a9b9dae619b132ac5f336f70d0b4ca"} Jul 09 04:08:06.517364 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:06.517329 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" event={"ID":"457b866c-b844-4825-b6da-19440a29b601","Type":"ContainerStarted","Data":"e86bdaf69c46189e93e230fc70e2da99b65b1ca6702639906547db0d123edc07"} Jul 09 04:08:06.539288 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:06.539237 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wqxhf" podStartSLOduration=7.820847726 podStartE2EDuration="43.539222342s" totalImagesPullingTime="35.718374616s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:06.53775571 +0000 UTC m=+43.774725887" watchObservedRunningTime="2026-07-09 04:08:06.539222342 +0000 UTC m=+43.776192520" Jul 09 04:08:06.936336 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:06.936299 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:08:06.936505 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:06.936425 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:08:06.936505 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:06.936486 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:08:14.936468478 +0000 UTC m=+52.173438659 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:08:14.992295 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:14.992251 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:08:14.992748 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:14.992379 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:08:14.992748 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:14.992433 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:08:30.99242019 +0000 UTC m=+68.229390345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:08:24.485890 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:24.485857 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b5qk6" Jul 09 04:08:27.984785 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:27.984728 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:08:27.984785 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:27.984789 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:08:27.987694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:27.987670 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 04:08:27.987773 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:27.987740 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 04:08:27.995847 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:27.995820 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 04:08:27.995921 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:27.995896 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:09:31.995878094 +0000 UTC m=+129.232848251 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : secret "metrics-daemon-secret" not found Jul 09 04:08:27.998500 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:27.998467 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/10a1edc9-d06f-4cf7-b217-9334547578b8-original-pull-secret\") pod \"global-pull-secret-syncer-4dwrv\" (UID: \"10a1edc9-d06f-4cf7-b217-9334547578b8\") " pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:08:28.085331 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.085287 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:08:28.088031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.088012 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 04:08:28.098350 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.098324 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 04:08:28.108847 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.108816 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwrfc\" (UniqueName: \"kubernetes.io/projected/8d7ee2e8-b9cf-4480-9cc7-9922be49c262-kube-api-access-wwrfc\") pod \"network-check-target-ldjmc\" (UID: \"8d7ee2e8-b9cf-4480-9cc7-9922be49c262\") " pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:08:28.136701 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.136658 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4dwrv" Jul 09 04:08:28.146418 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.146391 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-kh5dc\"" Jul 09 04:08:28.154756 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.154728 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:08:28.287377 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.287340 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4dwrv"] Jul 09 04:08:28.291135 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:08:28.291104 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a1edc9_d06f_4cf7_b217_9334547578b8.slice/crio-e38cb5493250a74739367daa60ee7b7504bdab23a2eb95d71f1114a249acc5c4 WatchSource:0}: Error finding container e38cb5493250a74739367daa60ee7b7504bdab23a2eb95d71f1114a249acc5c4: Status 404 returned error can't find the container with id e38cb5493250a74739367daa60ee7b7504bdab23a2eb95d71f1114a249acc5c4 Jul 09 04:08:28.309660 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.309633 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ldjmc"] Jul 09 04:08:28.312682 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:08:28.312638 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d7ee2e8_b9cf_4480_9cc7_9922be49c262.slice/crio-4577a6abecf25706f38f6309d8d71ab247e968a378d9a296d6a6b760daff0bff WatchSource:0}: Error finding container 4577a6abecf25706f38f6309d8d71ab247e968a378d9a296d6a6b760daff0bff: Status 404 returned error can't find the container with id 4577a6abecf25706f38f6309d8d71ab247e968a378d9a296d6a6b760daff0bff Jul 09 04:08:28.570983 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.570886 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4dwrv" event={"ID":"10a1edc9-d06f-4cf7-b217-9334547578b8","Type":"ContainerStarted","Data":"e38cb5493250a74739367daa60ee7b7504bdab23a2eb95d71f1114a249acc5c4"} Jul 09 04:08:28.571962 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:28.571927 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ldjmc" event={"ID":"8d7ee2e8-b9cf-4480-9cc7-9922be49c262","Type":"ContainerStarted","Data":"4577a6abecf25706f38f6309d8d71ab247e968a378d9a296d6a6b760daff0bff"} Jul 09 04:08:31.004212 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:31.004164 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:08:31.004706 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:31.004301 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:08:31.004706 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:08:31.004379 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:09:03.004359351 +0000 UTC m=+100.241329508 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:08:33.586996 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:33.586955 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4dwrv" event={"ID":"10a1edc9-d06f-4cf7-b217-9334547578b8","Type":"ContainerStarted","Data":"e5ac113fc49d363fc7341a2b307771db402133bc97cef0d3a95f8e11fc4f65ce"} Jul 09 04:08:33.588454 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:33.588430 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ldjmc" event={"ID":"8d7ee2e8-b9cf-4480-9cc7-9922be49c262","Type":"ContainerStarted","Data":"1031b498811bd789e8c70d3e38f74d28a431b065f540d52cdde12f13a52b5686"} Jul 09 04:08:33.588583 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:33.588570 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:08:33.601102 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:33.601056 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4dwrv" podStartSLOduration=64.846299918 podStartE2EDuration="1m9.601042067s" totalImagesPullingTime="4.754742149s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:07:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:33.600402944 +0000 UTC m=+70.837373123" watchObservedRunningTime="2026-07-09 04:08:33.601042067 +0000 UTC m=+70.838012246" Jul 09 04:08:33.614318 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:33.614266 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ldjmc" podStartSLOduration=65.885686708 podStartE2EDuration="1m10.614251705s" totalImagesPullingTime="4.728564997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:33.613712846 +0000 UTC m=+70.850683025" watchObservedRunningTime="2026-07-09 04:08:33.614251705 +0000 UTC m=+70.851221886" Jul 09 04:08:51.940169 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.940131 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95"] Jul 09 04:08:51.943920 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.943890 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:51.947732 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.947697 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 04:08:51.947879 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.947736 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 04:08:51.947879 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.947750 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 04:08:51.947879 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.947807 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 04:08:51.948944 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.948922 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 04:08:51.949053 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.948971 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 04:08:51.949053 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.949016 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 04:08:51.955682 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:51.955649 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95"] Jul 09 04:08:52.038784 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.038743 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-hub\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.038784 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.038786 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzdv9\" (UniqueName: \"kubernetes.io/projected/5d12b261-fd6c-4955-a668-3c01b5140541-kube-api-access-gzdv9\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.039021 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.038815 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.039021 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.038860 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-ca\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.039021 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.038876 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.039021 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.038892 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d12b261-fd6c-4955-a668-3c01b5140541-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.140004 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.139944 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-hub\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.140004 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.140010 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzdv9\" (UniqueName: \"kubernetes.io/projected/5d12b261-fd6c-4955-a668-3c01b5140541-kube-api-access-gzdv9\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.140271 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.140055 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.140271 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.140083 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-ca\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.140271 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.140105 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.140271 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.140176 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d12b261-fd6c-4955-a668-3c01b5140541-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.141169 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.141137 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d12b261-fd6c-4955-a668-3c01b5140541-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.142807 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.142781 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-ca\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.142968 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.142949 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.143108 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.143087 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-hub\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.143145 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.143112 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5d12b261-fd6c-4955-a668-3c01b5140541-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.148628 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.148604 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzdv9\" (UniqueName: \"kubernetes.io/projected/5d12b261-fd6c-4955-a668-3c01b5140541-kube-api-access-gzdv9\") pod \"cluster-proxy-proxy-agent-5b95cbb68b-4wh95\" (UID: \"5d12b261-fd6c-4955-a668-3c01b5140541\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.271564 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.271446 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:08:52.401189 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.401160 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95"] Jul 09 04:08:52.404465 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:08:52.404439 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d12b261_fd6c_4955_a668_3c01b5140541.slice/crio-82738a67d4a5c3cbb065c823220c58674d869f41df6056d06af0c00c11b8a78a WatchSource:0}: Error finding container 82738a67d4a5c3cbb065c823220c58674d869f41df6056d06af0c00c11b8a78a: Status 404 returned error can't find the container with id 82738a67d4a5c3cbb065c823220c58674d869f41df6056d06af0c00c11b8a78a Jul 09 04:08:52.635873 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:52.635839 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" event={"ID":"5d12b261-fd6c-4955-a668-3c01b5140541","Type":"ContainerStarted","Data":"82738a67d4a5c3cbb065c823220c58674d869f41df6056d06af0c00c11b8a78a"} Jul 09 04:08:56.651951 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:56.651913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" event={"ID":"5d12b261-fd6c-4955-a668-3c01b5140541","Type":"ContainerStarted","Data":"c491c7414226a3468731c406c85c675486c0eeefa2be919739d288d77ad3e3f3"} Jul 09 04:08:56.652367 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:56.651967 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" event={"ID":"5d12b261-fd6c-4955-a668-3c01b5140541","Type":"ContainerStarted","Data":"559becc31b763ecf148e7e7be6d0d25c40ef1c9102fa151466639f4aaf631f6a"} Jul 09 04:08:56.652367 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:56.651985 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" event={"ID":"5d12b261-fd6c-4955-a668-3c01b5140541","Type":"ContainerStarted","Data":"5fd69587d311cdf47eeb01f7de5a9d7eb0fff46ea9ac5a2d4ccc3a2697a01540"} Jul 09 04:08:56.670014 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:08:56.669955 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" podStartSLOduration=1.714709796 podStartE2EDuration="5.669940055s" totalImagesPullingTime="3.955230259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:08:56.66841615 +0000 UTC m=+93.905386339" watchObservedRunningTime="2026-07-09 04:08:56.669940055 +0000 UTC m=+93.906910211" Jul 09 04:09:03.023987 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:09:03.023943 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:09:03.024453 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:09:03.024127 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:09:03.024453 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:09:03.024211 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:10:07.024193529 +0000 UTC m=+164.261163688 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:09:04.593444 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:09:04.593415 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ldjmc" Jul 09 04:09:32.029198 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:09:32.029144 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:09:32.029729 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:09:32.029297 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 04:09:32.029729 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:09:32.029370 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:34.029349604 +0000 UTC m=+251.266319780 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : secret "metrics-daemon-secret" not found Jul 09 04:10:02.145664 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:10:02.145599 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2q2pz" podUID="66216ebf-6785-49d8-acf3-ba7de5e1ef2c" Jul 09 04:10:02.838231 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:10:02.838201 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2q2pz" Jul 09 04:10:03.250546 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:10:03.250477 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:10:07.074189 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:10:07.074128 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:10:07.074645 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:10:07.074289 2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 04:10:07.074645 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:10:07.074361 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls podName:66216ebf-6785-49d8-acf3-ba7de5e1ef2c nodeName:}" failed. No retries permitted until 2026-07-09 04:12:09.074343709 +0000 UTC m=+286.311313864 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls") pod "dns-default-2q2pz" (UID: "66216ebf-6785-49d8-acf3-ba7de5e1ef2c") : secret "dns-default-metrics-tls" not found Jul 09 04:10:16.225203 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:10:16.225097 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:11:19.015428 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.015390 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn"] Jul 09 04:11:19.018118 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.018096 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.020730 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.020700 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 04:11:19.020930 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.020914 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 04:11:19.023889 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.023866 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 04:11:19.024014 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.023912 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 04:11:19.024014 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.023928 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9k7v5\"" Jul 09 04:11:19.029344 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.029319 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn"] Jul 09 04:11:19.078752 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.078710 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.078752 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.078757 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.078964 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.078843 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt8dx\" (UniqueName: \"kubernetes.io/projected/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-kube-api-access-gt8dx\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.180041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.179984 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt8dx\" (UniqueName: \"kubernetes.io/projected/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-kube-api-access-gt8dx\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.180203 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.180073 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.180203 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.180097 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.180319 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:19.180242 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:19.180369 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:19.180332 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls podName:60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:19.680310428 +0000 UTC m=+236.917280588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-94dzn" (UID: "60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:19.180743 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.180722 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.190226 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.190207 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt8dx\" (UniqueName: \"kubernetes.io/projected/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-kube-api-access-gt8dx\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.684368 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:19.684328 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:19.684551 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:19.684477 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:19.684605 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:19.684558 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls podName:60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:20.684539216 +0000 UTC m=+237.921509394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-94dzn" (UID: "60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:20.690380 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:20.690335 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:20.690817 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:20.690455 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:20.690817 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:20.690542 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls podName:60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:22.690507326 +0000 UTC m=+239.927477481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-94dzn" (UID: "60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:22.706460 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:22.706413 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:22.706892 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:22.706591 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:22.706892 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:22.706669 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls podName:60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:26.706652237 +0000 UTC m=+243.943622393 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-94dzn" (UID: "60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:24.910108 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.910075 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-t52jt"] Jul 09 04:11:24.913199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.913173 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:24.916398 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.916368 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 04:11:24.916741 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.916717 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 04:11:24.916828 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.916745 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 04:11:24.916828 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.916808 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 04:11:24.918388 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.918368 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c6c2f\"" Jul 09 04:11:24.929114 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:24.929077 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t52jt"] Jul 09 04:11:25.026914 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.026859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dba29411-249a-4b04-82ef-7de59d15128d-data-volume\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.027115 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.026964 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dba29411-249a-4b04-82ef-7de59d15128d-crio-socket\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.027115 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.027031 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.027115 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.027066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.027115 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.027092 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8f5d\" (UniqueName: \"kubernetes.io/projected/dba29411-249a-4b04-82ef-7de59d15128d-kube-api-access-c8f5d\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.128344 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.128306 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dba29411-249a-4b04-82ef-7de59d15128d-data-volume\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.128344 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.128361 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dba29411-249a-4b04-82ef-7de59d15128d-crio-socket\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.128381 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.128397 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.128412 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8f5d\" (UniqueName: \"kubernetes.io/projected/dba29411-249a-4b04-82ef-7de59d15128d-kube-api-access-c8f5d\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.128469 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.128487 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dba29411-249a-4b04-82ef-7de59d15128d-crio-socket\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.128572 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.128588 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:25.628564489 +0000 UTC m=+242.865534665 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : configmap "kube-rbac-proxy" not found Jul 09 04:11:25.128647 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.128636 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:25.62861385 +0000 UTC m=+242.865584006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:25.128912 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.128762 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dba29411-249a-4b04-82ef-7de59d15128d-data-volume\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.136880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.136852 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8f5d\" (UniqueName: \"kubernetes.io/projected/dba29411-249a-4b04-82ef-7de59d15128d-kube-api-access-c8f5d\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.633670 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.633630 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.633670 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:25.633671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:25.633879 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.633752 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:25.633879 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.633779 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:25.633879 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.633826 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:26.633807915 +0000 UTC m=+243.870778071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : configmap "kube-rbac-proxy" not found Jul 09 04:11:25.633879 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:25.633840 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:26.633834272 +0000 UTC m=+243.870804428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:26.640774 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:26.640719 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:26.640774 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:26.640771 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:26.641206 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:26.640852 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:26.641206 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:26.640869 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:26.641206 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:26.640920 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:28.640906555 +0000 UTC m=+245.877876711 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:26.641206 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:26.640936 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:28.640929494 +0000 UTC m=+245.877899650 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : configmap "kube-rbac-proxy" not found Jul 09 04:11:26.741886 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:26.741841 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:26.742006 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:26.741987 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:26.742064 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:26.742054 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls podName:60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:34.742037569 +0000 UTC m=+251.979007725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-94dzn" (UID: "60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:26.889977 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:26.889950 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bcknn_e5450f12-4d48-4380-b616-623f5b43eaa6/dns-node-resolver/0.log" Jul 09 04:11:27.351169 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.351138 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vv58w"] Jul 09 04:11:27.354116 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.354098 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.356860 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.356830 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 04:11:27.356998 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.356830 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 04:11:27.356998 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.356876 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 04:11:27.356998 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.356899 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 04:11:27.358047 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.358031 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rh8dn\"" Jul 09 04:11:27.361277 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.361251 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vv58w"] Jul 09 04:11:27.446066 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.446025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ef151fec-0a45-4d5d-bebc-060929f0e118-signing-cabundle\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.446066 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.446065 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vchn7\" (UniqueName: \"kubernetes.io/projected/ef151fec-0a45-4d5d-bebc-060929f0e118-kube-api-access-vchn7\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.446347 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.446206 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ef151fec-0a45-4d5d-bebc-060929f0e118-signing-key\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.547206 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.547169 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ef151fec-0a45-4d5d-bebc-060929f0e118-signing-key\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.547206 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.547213 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ef151fec-0a45-4d5d-bebc-060929f0e118-signing-cabundle\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.547405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.547229 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vchn7\" (UniqueName: \"kubernetes.io/projected/ef151fec-0a45-4d5d-bebc-060929f0e118-kube-api-access-vchn7\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.547903 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.547877 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ef151fec-0a45-4d5d-bebc-060929f0e118-signing-cabundle\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.549840 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.549817 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ef151fec-0a45-4d5d-bebc-060929f0e118-signing-key\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.555324 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.555300 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vchn7\" (UniqueName: \"kubernetes.io/projected/ef151fec-0a45-4d5d-bebc-060929f0e118-kube-api-access-vchn7\") pod \"service-ca-758f599c5-vv58w\" (UID: \"ef151fec-0a45-4d5d-bebc-060929f0e118\") " pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.664105 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.664071 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-vv58w" Jul 09 04:11:27.689334 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.689303 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5wl45_b77cdd8d-939d-4119-9384-38c48868d305/node-ca/0.log" Jul 09 04:11:27.791836 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:27.791801 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-vv58w"] Jul 09 04:11:27.795453 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:11:27.795416 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef151fec_0a45_4d5d_bebc_060929f0e118.slice/crio-d334115fa8663f722cefc71ef05334b5211835485272354127d9f9b846e69e82 WatchSource:0}: Error finding container d334115fa8663f722cefc71ef05334b5211835485272354127d9f9b846e69e82: Status 404 returned error can't find the container with id d334115fa8663f722cefc71ef05334b5211835485272354127d9f9b846e69e82 Jul 09 04:11:28.075279 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:28.075191 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-vv58w" event={"ID":"ef151fec-0a45-4d5d-bebc-060929f0e118","Type":"ContainerStarted","Data":"d334115fa8663f722cefc71ef05334b5211835485272354127d9f9b846e69e82"} Jul 09 04:11:28.488726 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:28.488693 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-78klw_c3d7be2b-3556-4dda-b2fb-d7ed0c822414/serve-healthcheck-canary/0.log" Jul 09 04:11:28.656272 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:28.656227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:28.656448 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:28.656280 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:28.656448 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:28.656360 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:28.656448 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:28.656433 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:32.656415259 +0000 UTC m=+249.893385416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : configmap "kube-rbac-proxy" not found Jul 09 04:11:28.656650 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:28.656474 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:28.656650 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:28.656565 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:32.656545782 +0000 UTC m=+249.893515945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:30.082235 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:30.082194 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-vv58w" event={"ID":"ef151fec-0a45-4d5d-bebc-060929f0e118","Type":"ContainerStarted","Data":"9784662f0e8d158de8dacfbd8786f1636ea46231e7f322b3d222e1ee0e042243"} Jul 09 04:11:30.097428 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:30.097382 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-vv58w" podStartSLOduration=1.284215657 podStartE2EDuration="3.097367882s" totalImagesPullingTime="1.813152225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:30.0968506 +0000 UTC m=+247.333820779" watchObservedRunningTime="2026-07-09 04:11:30.097367882 +0000 UTC m=+247.334338060" Jul 09 04:11:32.692842 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:32.692783 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:32.692842 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:32.692842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:32.693357 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:32.692914 2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 04:11:32.693357 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:32.692983 2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 04:11:32.693357 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:32.692989 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:40.692973531 +0000 UTC m=+257.929943686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : configmap "kube-rbac-proxy" not found Jul 09 04:11:32.693357 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:32.693062 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls podName:dba29411-249a-4b04-82ef-7de59d15128d nodeName:}" failed. No retries permitted until 2026-07-09 04:11:40.693050417 +0000 UTC m=+257.930020573 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-t52jt" (UID: "dba29411-249a-4b04-82ef-7de59d15128d") : secret "insights-runtime-extractor-tls" not found Jul 09 04:11:34.103270 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:34.103232 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:11:34.103695 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:34.103395 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 09 04:11:34.103695 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:34.103465 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs podName:65b7ad76-b3e5-48b9-abcc-a556fbc1e905 nodeName:}" failed. No retries permitted until 2026-07-09 04:13:36.103445263 +0000 UTC m=+373.340415427 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs") pod "network-metrics-daemon-vqlpv" (UID: "65b7ad76-b3e5-48b9-abcc-a556fbc1e905") : secret "metrics-daemon-secret" not found Jul 09 04:11:34.811486 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:34.811443 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:34.811688 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:34.811583 2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:34.811688 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:34.811657 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls podName:60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:50.811643076 +0000 UTC m=+268.048613232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-94dzn" (UID: "60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3") : secret "cluster-monitoring-operator-tls" not found Jul 09 04:11:40.758175 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:40.758138 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:40.758175 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:40.758178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:40.758734 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:40.758710 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dba29411-249a-4b04-82ef-7de59d15128d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:40.760683 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:40.760655 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dba29411-249a-4b04-82ef-7de59d15128d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-t52jt\" (UID: \"dba29411-249a-4b04-82ef-7de59d15128d\") " pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:40.823472 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:40.823431 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-t52jt" Jul 09 04:11:40.950254 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:40.950220 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-t52jt"] Jul 09 04:11:40.953222 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:11:40.953183 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba29411_249a_4b04_82ef_7de59d15128d.slice/crio-069164033fb5ca025f3156e0545d9505054589babebe50c496f1722de0bf4aaf WatchSource:0}: Error finding container 069164033fb5ca025f3156e0545d9505054589babebe50c496f1722de0bf4aaf: Status 404 returned error can't find the container with id 069164033fb5ca025f3156e0545d9505054589babebe50c496f1722de0bf4aaf Jul 09 04:11:41.117087 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:41.117003 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t52jt" event={"ID":"dba29411-249a-4b04-82ef-7de59d15128d","Type":"ContainerStarted","Data":"069164033fb5ca025f3156e0545d9505054589babebe50c496f1722de0bf4aaf"} Jul 09 04:11:45.131162 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:45.131072 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t52jt" event={"ID":"dba29411-249a-4b04-82ef-7de59d15128d","Type":"ContainerStarted","Data":"ea56d5db16d0b05bb6dd7201c6f5c5df318ba367acda0a4f6e27db8efa9a0222"} Jul 09 04:11:46.136153 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:46.136114 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t52jt" event={"ID":"dba29411-249a-4b04-82ef-7de59d15128d","Type":"ContainerStarted","Data":"cfa7b56a4789270b447cdbb8cd5886c0399ba9004896abe313657ee970bbd84a"} Jul 09 04:11:48.144843 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:48.144813 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-t52jt" event={"ID":"dba29411-249a-4b04-82ef-7de59d15128d","Type":"ContainerStarted","Data":"89d7e29af28bc74ab35c1cd8a812ae357e279b3e3d53b64fc4d9c8fb2c041a6d"} Jul 09 04:11:48.161498 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:48.161442 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-t52jt" podStartSLOduration=17.211882635 podStartE2EDuration="24.161425579s" totalImagesPullingTime="6.949542944s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:11:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:48.160655317 +0000 UTC m=+265.397625496" watchObservedRunningTime="2026-07-09 04:11:48.161425579 +0000 UTC m=+265.398395757" Jul 09 04:11:50.837032 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:50.836997 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:50.839498 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:50.839472 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-94dzn\" (UID: \"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:51.129569 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:51.129461 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9k7v5\"" Jul 09 04:11:51.137688 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:51.137661 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" Jul 09 04:11:51.265973 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:51.265944 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn"] Jul 09 04:11:51.269076 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:11:51.269045 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c1f4a4_fa64_4ce8_ae82_1bcdb46bf4e3.slice/crio-b0ff23519d43c6564a9ddf6fe6abb43b9ce75535e53eb68c418467b2e588a137 WatchSource:0}: Error finding container b0ff23519d43c6564a9ddf6fe6abb43b9ce75535e53eb68c418467b2e588a137: Status 404 returned error can't find the container with id b0ff23519d43c6564a9ddf6fe6abb43b9ce75535e53eb68c418467b2e588a137 Jul 09 04:11:52.157818 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:52.157783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" event={"ID":"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3","Type":"ContainerStarted","Data":"b0ff23519d43c6564a9ddf6fe6abb43b9ce75535e53eb68c418467b2e588a137"} Jul 09 04:11:52.273199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:52.273153 2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" podUID="5d12b261-fd6c-4955-a668-3c01b5140541" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 04:11:53.856700 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:53.856601 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw"] Jul 09 04:11:53.860170 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:53.860148 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:53.862622 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:53.862593 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 09 04:11:53.862749 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:53.862597 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-bkzkj\"" Jul 09 04:11:53.866444 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:53.866415 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw"] Jul 09 04:11:53.960666 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:53.960607 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8694deb3-5198-4f95-934e-4665d56408a9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-6ssvw\" (UID: \"8694deb3-5198-4f95-934e-4665d56408a9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:54.061315 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:54.061268 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8694deb3-5198-4f95-934e-4665d56408a9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-6ssvw\" (UID: \"8694deb3-5198-4f95-934e-4665d56408a9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:54.061488 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:54.061448 2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 09 04:11:54.061565 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:11:54.061553 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8694deb3-5198-4f95-934e-4665d56408a9-tls-certificates podName:8694deb3-5198-4f95-934e-4665d56408a9 nodeName:}" failed. No retries permitted until 2026-07-09 04:11:54.561509534 +0000 UTC m=+271.798479699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/8694deb3-5198-4f95-934e-4665d56408a9-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-6ssvw" (UID: "8694deb3-5198-4f95-934e-4665d56408a9") : secret "prometheus-operator-admission-webhook-tls" not found Jul 09 04:11:54.166859 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:54.166817 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" event={"ID":"60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3","Type":"ContainerStarted","Data":"784ec50e4509e980d3ef6f60aac9a72f6777ef4fb2add1822230645c75063adb"} Jul 09 04:11:54.182229 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:54.182169 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-94dzn" podStartSLOduration=34.064259245 podStartE2EDuration="36.182150604s" totalImagesPullingTime="2.117891359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:54.181164344 +0000 UTC m=+271.418134521" watchObservedRunningTime="2026-07-09 04:11:54.182150604 +0000 UTC m=+271.419120781" Jul 09 04:11:54.564837 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:54.564731 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8694deb3-5198-4f95-934e-4665d56408a9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-6ssvw\" (UID: \"8694deb3-5198-4f95-934e-4665d56408a9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:54.567448 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:54.567409 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8694deb3-5198-4f95-934e-4665d56408a9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-6ssvw\" (UID: \"8694deb3-5198-4f95-934e-4665d56408a9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:54.770925 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:54.770881 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:54.909123 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:54.909091 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw"] Jul 09 04:11:54.912114 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:11:54.912084 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8694deb3_5198_4f95_934e_4665d56408a9.slice/crio-6bb001794122afdacf584f21e36c6004a5e9f0e9428f70874b8857e5b8a8d440 WatchSource:0}: Error finding container 6bb001794122afdacf584f21e36c6004a5e9f0e9428f70874b8857e5b8a8d440: Status 404 returned error can't find the container with id 6bb001794122afdacf584f21e36c6004a5e9f0e9428f70874b8857e5b8a8d440 Jul 09 04:11:55.171111 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.171076 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" event={"ID":"8694deb3-5198-4f95-934e-4665d56408a9","Type":"ContainerStarted","Data":"6bb001794122afdacf584f21e36c6004a5e9f0e9428f70874b8857e5b8a8d440"} Jul 09 04:11:55.533619 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.533505 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8cb5498d7-2h52f"] Jul 09 04:11:55.536711 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.536680 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.539635 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.539594 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 04:11:55.541257 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.540975 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 04:11:55.541257 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.541029 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 04:11:55.541257 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.540981 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 04:11:55.541257 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.540982 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jw9kp\"" Jul 09 04:11:55.541257 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.540975 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 04:11:55.541257 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.541025 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 04:11:55.541257 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.541032 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 04:11:55.546335 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.546311 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cb5498d7-2h52f"] Jul 09 04:11:55.573304 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.573272 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-config\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.573304 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.573307 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6km9r\" (UniqueName: \"kubernetes.io/projected/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-kube-api-access-6km9r\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.573304 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.573344 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-oauth-config\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.573684 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.573398 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-service-ca\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.573684 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.573463 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-serving-cert\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.573684 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.573629 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-oauth-serving-cert\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.674849 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.674811 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-oauth-serving-cert\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675047 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.674877 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-config\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675047 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.674925 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6km9r\" (UniqueName: \"kubernetes.io/projected/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-kube-api-access-6km9r\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675047 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.674961 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-oauth-config\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675047 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.674976 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-service-ca\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675047 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.675037 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-serving-cert\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675752 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.675722 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-oauth-serving-cert\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675871 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.675745 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-service-ca\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.675871 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.675790 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-config\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.677885 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.677855 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-oauth-config\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.678067 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.678045 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-serving-cert\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.683185 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.683141 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6km9r\" (UniqueName: \"kubernetes.io/projected/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-kube-api-access-6km9r\") pod \"console-8cb5498d7-2h52f\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.848516 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.848420 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:11:55.993117 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:55.992953 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cb5498d7-2h52f"] Jul 09 04:11:55.996364 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:11:55.996333 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46ffe3b2_07c8_43cc_9a4f_c755246e5dfb.slice/crio-5f736873ae13866327b75e33cdff2984a18911561c2f91454866bf1660eca96e WatchSource:0}: Error finding container 5f736873ae13866327b75e33cdff2984a18911561c2f91454866bf1660eca96e: Status 404 returned error can't find the container with id 5f736873ae13866327b75e33cdff2984a18911561c2f91454866bf1660eca96e Jul 09 04:11:56.175789 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:56.175750 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" event={"ID":"8694deb3-5198-4f95-934e-4665d56408a9","Type":"ContainerStarted","Data":"0aaf89a8e717a45a61c2356710c12efae5abf32bce5ed69facd7adcbc3d07d5d"} Jul 09 04:11:56.176013 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:56.175909 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:56.177036 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:56.177014 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cb5498d7-2h52f" event={"ID":"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb","Type":"ContainerStarted","Data":"5f736873ae13866327b75e33cdff2984a18911561c2f91454866bf1660eca96e"} Jul 09 04:11:56.180623 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:56.180600 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" Jul 09 04:11:56.191320 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:11:56.191271 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-6ssvw" podStartSLOduration=2.187088597 podStartE2EDuration="3.191256955s" totalImagesPullingTime="1.004168358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:11:56.190562688 +0000 UTC m=+273.427532867" watchObservedRunningTime="2026-07-09 04:11:56.191256955 +0000 UTC m=+273.428227133" Jul 09 04:12:00.192916 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:00.192880 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cb5498d7-2h52f" event={"ID":"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb","Type":"ContainerStarted","Data":"e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699"} Jul 09 04:12:00.209110 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:00.209054 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8cb5498d7-2h52f" podStartSLOduration=1.9098612689999999 podStartE2EDuration="5.209038083s" totalImagesPullingTime="3.299176814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:11:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:00.208819411 +0000 UTC m=+277.445789590" watchObservedRunningTime="2026-07-09 04:12:00.209038083 +0000 UTC m=+277.446008325" Jul 09 04:12:00.926365 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:00.926335 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76c76dd6b7-7mx4q"] Jul 09 04:12:00.929366 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:00.929340 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:00.937669 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:00.937649 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 04:12:00.940593 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:00.940454 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76c76dd6b7-7mx4q"] Jul 09 04:12:01.019973 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.019934 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-trusted-ca-bundle\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.020164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.020006 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-serving-cert\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.020164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.020033 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-oauth-config\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.020164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.020048 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-service-ca\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.020164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.020073 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-config\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.020164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.020121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-oauth-serving-cert\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.020164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.020157 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jwmx\" (UniqueName: \"kubernetes.io/projected/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-kube-api-access-7jwmx\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.121302 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.121265 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-trusted-ca-bundle\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.121535 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.121346 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-serving-cert\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.121535 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.121372 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-oauth-config\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.121535 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.121395 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-service-ca\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.121535 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.121424 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-config\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.121535 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.121450 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-oauth-serving-cert\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.121535 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.121497 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jwmx\" (UniqueName: \"kubernetes.io/projected/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-kube-api-access-7jwmx\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.122387 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.122160 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-service-ca\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.122387 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.122239 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-config\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.122387 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.122247 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-oauth-serving-cert\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.122644 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.122430 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-trusted-ca-bundle\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.124042 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.124022 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-oauth-config\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.124164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.124145 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-serving-cert\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.131350 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.131322 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jwmx\" (UniqueName: \"kubernetes.io/projected/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-kube-api-access-7jwmx\") pod \"console-76c76dd6b7-7mx4q\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.238673 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.238594 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:01.372331 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:01.372297 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76c76dd6b7-7mx4q"] Jul 09 04:12:01.375457 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:12:01.375429 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0b086b_d51b_452a_aa81_9d62ec6feaa9.slice/crio-3cb2f21a96f99478d53bced2daf789e09419d7b615d07fd04fae05e1550028b9 WatchSource:0}: Error finding container 3cb2f21a96f99478d53bced2daf789e09419d7b615d07fd04fae05e1550028b9: Status 404 returned error can't find the container with id 3cb2f21a96f99478d53bced2daf789e09419d7b615d07fd04fae05e1550028b9 Jul 09 04:12:02.202957 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.202910 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76c76dd6b7-7mx4q" event={"ID":"5e0b086b-d51b-452a-aa81-9d62ec6feaa9","Type":"ContainerStarted","Data":"ef916ce9a0ff958b0ffbba368b087f669d074022e0715194a94568f9a7f1874d"} Jul 09 04:12:02.203148 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.203023 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76c76dd6b7-7mx4q" event={"ID":"5e0b086b-d51b-452a-aa81-9d62ec6feaa9","Type":"ContainerStarted","Data":"3cb2f21a96f99478d53bced2daf789e09419d7b615d07fd04fae05e1550028b9"} Jul 09 04:12:02.224283 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.224224 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76c76dd6b7-7mx4q" podStartSLOduration=2.224203561 podStartE2EDuration="2.224203561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:02.222732424 +0000 UTC m=+279.459702693" watchObservedRunningTime="2026-07-09 04:12:02.224203561 +0000 UTC m=+279.461173738" Jul 09 04:12:02.272803 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.272763 2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" podUID="5d12b261-fd6c-4955-a668-3c01b5140541" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 04:12:02.299013 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.298977 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-h6spx"] Jul 09 04:12:02.302715 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.302686 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.305082 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.305052 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 04:12:02.305398 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.305379 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 04:12:02.305492 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.305389 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 04:12:02.305492 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.305431 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 04:12:02.305625 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.305383 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rzmb5\"" Jul 09 04:12:02.332065 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332022 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7da38a96-1760-40b4-a54c-fd5356ecf2c7-metrics-client-ca\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332273 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332087 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-root\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332273 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332116 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-wtmp\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332273 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332150 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332273 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332196 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-textfile\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332273 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332220 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-tls\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332273 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332258 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-sys\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332584 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332283 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc78f\" (UniqueName: \"kubernetes.io/projected/7da38a96-1760-40b4-a54c-fd5356ecf2c7-kube-api-access-mc78f\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.332584 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.332345 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.432796 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.432740 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-textfile\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.432796 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.432796 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-tls\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433037 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.432823 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-sys\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433037 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.432848 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc78f\" (UniqueName: \"kubernetes.io/projected/7da38a96-1760-40b4-a54c-fd5356ecf2c7-kube-api-access-mc78f\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433037 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.432958 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-sys\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433192 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433035 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433192 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433118 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-textfile\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433192 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433129 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7da38a96-1760-40b4-a54c-fd5356ecf2c7-metrics-client-ca\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433195 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-root\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433222 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-wtmp\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433259 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433348 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433270 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-root\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433585 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433397 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-wtmp\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433789 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433767 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7da38a96-1760-40b4-a54c-fd5356ecf2c7-metrics-client-ca\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.433856 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.433770 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-accelerators-collector-config\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.435415 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.435393 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.435811 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.435789 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7da38a96-1760-40b4-a54c-fd5356ecf2c7-node-exporter-tls\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.441922 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.441889 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc78f\" (UniqueName: \"kubernetes.io/projected/7da38a96-1760-40b4-a54c-fd5356ecf2c7-kube-api-access-mc78f\") pod \"node-exporter-h6spx\" (UID: \"7da38a96-1760-40b4-a54c-fd5356ecf2c7\") " pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.613149 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:02.613055 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-h6spx" Jul 09 04:12:02.624602 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:12:02.624573 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da38a96_1760_40b4_a54c_fd5356ecf2c7.slice/crio-8d969ed399ea8bf360d8191105abab7204bff782edc11c6457dee42b6bb11ec4 WatchSource:0}: Error finding container 8d969ed399ea8bf360d8191105abab7204bff782edc11c6457dee42b6bb11ec4: Status 404 returned error can't find the container with id 8d969ed399ea8bf360d8191105abab7204bff782edc11c6457dee42b6bb11ec4 Jul 09 04:12:03.207575 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.207535 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6spx" event={"ID":"7da38a96-1760-40b4-a54c-fd5356ecf2c7","Type":"ContainerStarted","Data":"8d969ed399ea8bf360d8191105abab7204bff782edc11c6457dee42b6bb11ec4"} Jul 09 04:12:03.348940 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.348904 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:12:03.352923 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.352897 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.355590 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.355562 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 04:12:03.355897 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.355872 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 04:12:03.356155 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356045 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 04:12:03.356249 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356174 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 04:12:03.356249 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356188 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 04:12:03.356907 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356577 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 04:12:03.356907 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356603 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 04:12:03.356907 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356619 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 04:12:03.356907 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356768 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 04:12:03.356907 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.356857 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sgz4j\"" Jul 09 04:12:03.366989 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.366964 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:12:03.440427 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440381 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-out\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440646 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440431 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8skc\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-kube-api-access-c8skc\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440646 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440471 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440646 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440539 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440646 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440584 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440646 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440630 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440713 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440752 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-web-config\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440782 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440808 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.440905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440834 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.441163 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440928 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.441163 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.440982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.541648 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.541546 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.541648 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.541607 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-out\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.541648 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.541624 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8skc\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-kube-api-access-c8skc\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542067 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542136 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542169 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542214 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542270 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542299 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-web-config\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542329 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542360 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542385 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542448 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:12:03.542605 2584 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:12:03.542681 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls podName:9b0d4951-d154-431c-adb6-db1a18ec8b8e nodeName:}" failed. No retries permitted until 2026-07-09 04:12:04.042661421 +0000 UTC m=+281.279631582 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e") : secret "alertmanager-main-tls" not found Jul 09 04:12:03.543183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.542794 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.544102 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.543745 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.544102 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.543960 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.546955 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.546923 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.547426 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.547374 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-out\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.547743 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.547709 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.548261 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.548215 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.548761 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.548721 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.549581 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.549471 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.550321 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.549989 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-web-config\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.550321 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.550259 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-volume\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:03.552214 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:03.552153 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8skc\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-kube-api-access-c8skc\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:04.047901 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:04.047862 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:04.050469 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:04.050437 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:04.212467 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:04.212428 2584 generic.go:356] "Generic (PLEG): container finished" podID="7da38a96-1760-40b4-a54c-fd5356ecf2c7" containerID="5ee59e32a39465afc2e7828613bccd011b511bc413598897b2bd4c336c485850" exitCode=0 Jul 09 04:12:04.212684 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:04.212477 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6spx" event={"ID":"7da38a96-1760-40b4-a54c-fd5356ecf2c7","Type":"ContainerDied","Data":"5ee59e32a39465afc2e7828613bccd011b511bc413598897b2bd4c336c485850"} Jul 09 04:12:04.265748 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:04.265722 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:12:04.464501 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:04.464419 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:12:04.467867 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:12:04.467838 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b0d4951_d154_431c_adb6_db1a18ec8b8e.slice/crio-d2be8f2b09a000c2f1a6e3d457ce4281091e480a8629b6dd64d03880bfe8a116 WatchSource:0}: Error finding container d2be8f2b09a000c2f1a6e3d457ce4281091e480a8629b6dd64d03880bfe8a116: Status 404 returned error can't find the container with id d2be8f2b09a000c2f1a6e3d457ce4281091e480a8629b6dd64d03880bfe8a116 Jul 09 04:12:05.219106 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:05.219064 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerStarted","Data":"d2be8f2b09a000c2f1a6e3d457ce4281091e480a8629b6dd64d03880bfe8a116"} Jul 09 04:12:05.222406 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:05.222376 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6spx" event={"ID":"7da38a96-1760-40b4-a54c-fd5356ecf2c7","Type":"ContainerStarted","Data":"4aefcd041fa26807f89cb4b24b1d92b2eb640381ca1d46eacfd27caeb54e0185"} Jul 09 04:12:05.222970 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:05.222931 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-h6spx" event={"ID":"7da38a96-1760-40b4-a54c-fd5356ecf2c7","Type":"ContainerStarted","Data":"c8dd66660f69ee900310ef0390e99c7a517d7602772da102932b0f791aaba15e"} Jul 09 04:12:05.245574 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:05.245500 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-h6spx" podStartSLOduration=2.152152039 podStartE2EDuration="3.245479808s" totalImagesPullingTime="1.013662467s" totalInitContainerRuntime="79.665302ms" isStatefulPod=true podCreationTimestamp="2026-07-09 04:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:05.245313134 +0000 UTC m=+282.482283313" watchObservedRunningTime="2026-07-09 04:12:05.245479808 +0000 UTC m=+282.482449985" Jul 09 04:12:05.839646 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:12:05.839614 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2q2pz" podUID="66216ebf-6785-49d8-acf3-ba7de5e1ef2c" Jul 09 04:12:05.848928 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:05.848901 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:12:05.849044 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:05.848959 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:12:05.854907 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:05.854887 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:12:06.227401 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:06.227372 2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerID="dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357" exitCode=0 Jul 09 04:12:06.227603 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:06.227470 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357"} Jul 09 04:12:06.227955 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:06.227888 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2q2pz" Jul 09 04:12:06.231655 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:06.231635 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:12:09.096718 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.096688 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:12:09.099151 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.099088 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/66216ebf-6785-49d8-acf3-ba7de5e1ef2c-metrics-tls\") pod \"dns-default-2q2pz\" (UID: \"66216ebf-6785-49d8-acf3-ba7de5e1ef2c\") " pod="openshift-dns/dns-default-2q2pz" Jul 09 04:12:09.144730 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.144694 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76c76dd6b7-7mx4q"] Jul 09 04:12:09.175966 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.175934 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5df89db6f9-mlv5r"] Jul 09 04:12:09.177858 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.177841 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.189847 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.189824 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5df89db6f9-mlv5r"] Jul 09 04:12:09.230736 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.230707 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mbnmf\"" Jul 09 04:12:09.238551 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.238512 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2q2pz" Jul 09 04:12:09.241847 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.241820 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerStarted","Data":"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a"} Jul 09 04:12:09.241969 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.241897 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerStarted","Data":"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5"} Jul 09 04:12:09.241969 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.241937 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerStarted","Data":"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0"} Jul 09 04:12:09.242075 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.241971 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerStarted","Data":"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87"} Jul 09 04:12:09.242075 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.242012 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerStarted","Data":"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423"} Jul 09 04:12:09.299204 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.299120 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-config\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.299204 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.299168 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-service-ca\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.299418 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.299218 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crv7f\" (UniqueName: \"kubernetes.io/projected/28de9a4a-db2b-4b7f-a0e1-95dece215053-kube-api-access-crv7f\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.299418 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.299254 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-serving-cert\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.299418 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.299336 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-oauth-config\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.299625 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.299440 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-trusted-ca-bundle\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.299625 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.299510 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-oauth-serving-cert\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.366709 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.366632 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2q2pz"] Jul 09 04:12:09.369230 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:12:09.369201 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66216ebf_6785_49d8_acf3_ba7de5e1ef2c.slice/crio-859d58cfa923226bc52bdcfbb5c70241f14dfbbed724cfd3dd1ae0a2c9a63f17 WatchSource:0}: Error finding container 859d58cfa923226bc52bdcfbb5c70241f14dfbbed724cfd3dd1ae0a2c9a63f17: Status 404 returned error can't find the container with id 859d58cfa923226bc52bdcfbb5c70241f14dfbbed724cfd3dd1ae0a2c9a63f17 Jul 09 04:12:09.400778 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.400737 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-trusted-ca-bundle\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.400917 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.400819 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-oauth-serving-cert\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.400917 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.400861 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-config\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.400930 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-service-ca\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.400974 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crv7f\" (UniqueName: \"kubernetes.io/projected/28de9a4a-db2b-4b7f-a0e1-95dece215053-kube-api-access-crv7f\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.401010 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-serving-cert\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.401038 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-oauth-config\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401617 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.401595 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-oauth-serving-cert\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401723 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.401689 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-config\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401723 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.401697 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-trusted-ca-bundle\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.401798 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.401773 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-service-ca\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.403641 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.403609 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-serving-cert\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.403728 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.403710 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-oauth-config\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.411708 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.411688 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crv7f\" (UniqueName: \"kubernetes.io/projected/28de9a4a-db2b-4b7f-a0e1-95dece215053-kube-api-access-crv7f\") pod \"console-5df89db6f9-mlv5r\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.486330 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.486286 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:09.620420 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:09.620346 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5df89db6f9-mlv5r"] Jul 09 04:12:09.623151 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:12:09.623123 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28de9a4a_db2b_4b7f_a0e1_95dece215053.slice/crio-5f7df5ebc75ea3aec7c2243a512f75155ec52918353c999686d859557275489c WatchSource:0}: Error finding container 5f7df5ebc75ea3aec7c2243a512f75155ec52918353c999686d859557275489c: Status 404 returned error can't find the container with id 5f7df5ebc75ea3aec7c2243a512f75155ec52918353c999686d859557275489c Jul 09 04:12:10.251797 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:10.251753 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerStarted","Data":"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8"} Jul 09 04:12:10.253290 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:10.253262 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5df89db6f9-mlv5r" event={"ID":"28de9a4a-db2b-4b7f-a0e1-95dece215053","Type":"ContainerStarted","Data":"b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8"} Jul 09 04:12:10.253424 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:10.253307 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5df89db6f9-mlv5r" event={"ID":"28de9a4a-db2b-4b7f-a0e1-95dece215053","Type":"ContainerStarted","Data":"5f7df5ebc75ea3aec7c2243a512f75155ec52918353c999686d859557275489c"} Jul 09 04:12:10.254254 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:10.254235 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2q2pz" event={"ID":"66216ebf-6785-49d8-acf3-ba7de5e1ef2c","Type":"ContainerStarted","Data":"859d58cfa923226bc52bdcfbb5c70241f14dfbbed724cfd3dd1ae0a2c9a63f17"} Jul 09 04:12:10.305464 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:10.305352 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.43892075 podStartE2EDuration="7.305327928s" totalImagesPullingTime="4.848504947s" totalInitContainerRuntime="17.902231ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:10.285466121 +0000 UTC m=+287.522436302" watchObservedRunningTime="2026-07-09 04:12:10.305327928 +0000 UTC m=+287.542298108" Jul 09 04:12:10.305781 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:10.305751 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5df89db6f9-mlv5r" podStartSLOduration=1.305740577 podStartE2EDuration="1.305740577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:10.304061825 +0000 UTC m=+287.541032002" watchObservedRunningTime="2026-07-09 04:12:10.305740577 +0000 UTC m=+287.542710757" Jul 09 04:12:11.239676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:11.239592 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:11.259635 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:11.259596 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2q2pz" event={"ID":"66216ebf-6785-49d8-acf3-ba7de5e1ef2c","Type":"ContainerStarted","Data":"7a6b8a367b0bc0842629a1ac42ca211ffe18eab05f4166c61cc17b6e5a1c175c"} Jul 09 04:12:11.260028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:11.259657 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2q2pz" event={"ID":"66216ebf-6785-49d8-acf3-ba7de5e1ef2c","Type":"ContainerStarted","Data":"62aa80419ec3754fa80d2ed9ae573d84b80567b4cdd698b9d6beef9cb3d7b41e"} Jul 09 04:12:11.260028 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:11.259756 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2q2pz" Jul 09 04:12:11.279299 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:11.279234 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2q2pz" podStartSLOduration=250.674898854 podStartE2EDuration="4m12.279215397s" totalImagesPullingTime="1.604316543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:11.278681299 +0000 UTC m=+288.515651499" watchObservedRunningTime="2026-07-09 04:12:11.279215397 +0000 UTC m=+288.516185575" Jul 09 04:12:12.272931 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.272895 2584 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" podUID="5d12b261-fd6c-4955-a668-3c01b5140541" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 09 04:12:12.273302 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.272966 2584 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" Jul 09 04:12:12.273501 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.273468 2584 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c491c7414226a3468731c406c85c675486c0eeefa2be919739d288d77ad3e3f3"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 09 04:12:12.273564 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.273548 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" podUID="5d12b261-fd6c-4955-a668-3c01b5140541" containerName="service-proxy" containerID="cri-o://c491c7414226a3468731c406c85c675486c0eeefa2be919739d288d77ad3e3f3" gracePeriod=30 Jul 09 04:12:12.312718 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.312680 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5df89db6f9-mlv5r"] Jul 09 04:12:12.368217 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.368181 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f866bfd4d-hcqsn"] Jul 09 04:12:12.370378 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.370359 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.379660 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.379630 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f866bfd4d-hcqsn"] Jul 09 04:12:12.430199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.430170 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-trusted-ca-bundle\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.430350 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.430215 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-oauth-serving-cert\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.430350 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.430257 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-serving-cert\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.430350 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.430275 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-oauth-config\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.430350 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.430316 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr2dr\" (UniqueName: \"kubernetes.io/projected/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-kube-api-access-mr2dr\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.430609 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.430383 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-config\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.430609 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.430420 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-service-ca\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.531368 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.531264 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-oauth-serving-cert\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.531368 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.531318 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-serving-cert\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.531368 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.531335 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-oauth-config\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.531677 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.531382 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mr2dr\" (UniqueName: \"kubernetes.io/projected/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-kube-api-access-mr2dr\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.531677 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.531417 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-config\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.531677 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.531452 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-service-ca\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.531677 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.531639 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-trusted-ca-bundle\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.532162 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.532135 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-oauth-serving-cert\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.532416 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.532179 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-service-ca\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.532416 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.532190 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-config\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.532721 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.532694 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-trusted-ca-bundle\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.534133 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.534110 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-oauth-config\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.534304 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.534288 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-serving-cert\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.539763 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.539735 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr2dr\" (UniqueName: \"kubernetes.io/projected/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-kube-api-access-mr2dr\") pod \"console-f866bfd4d-hcqsn\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.680947 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.680913 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:12.812339 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:12.812314 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f866bfd4d-hcqsn"] Jul 09 04:12:12.814709 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:12:12.814664 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2d01ed_df03_41ea_8ca1_2e85d9231e07.slice/crio-c46f2154bf4613fdd0f560dddd52c594b7c0cbc96b6966d65e250211787a49bd WatchSource:0}: Error finding container c46f2154bf4613fdd0f560dddd52c594b7c0cbc96b6966d65e250211787a49bd: Status 404 returned error can't find the container with id c46f2154bf4613fdd0f560dddd52c594b7c0cbc96b6966d65e250211787a49bd Jul 09 04:12:13.269671 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:13.269631 2584 generic.go:356] "Generic (PLEG): container finished" podID="5d12b261-fd6c-4955-a668-3c01b5140541" containerID="c491c7414226a3468731c406c85c675486c0eeefa2be919739d288d77ad3e3f3" exitCode=2 Jul 09 04:12:13.269849 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:13.269709 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" event={"ID":"5d12b261-fd6c-4955-a668-3c01b5140541","Type":"ContainerDied","Data":"c491c7414226a3468731c406c85c675486c0eeefa2be919739d288d77ad3e3f3"} Jul 09 04:12:13.269849 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:13.269753 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5b95cbb68b-4wh95" event={"ID":"5d12b261-fd6c-4955-a668-3c01b5140541","Type":"ContainerStarted","Data":"df02bc519fba40885217ea869a102330491a79de6d651249e7073fd82e3f3988"} Jul 09 04:12:13.271281 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:13.271254 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f866bfd4d-hcqsn" event={"ID":"cb2d01ed-df03-41ea-8ca1-2e85d9231e07","Type":"ContainerStarted","Data":"ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75"} Jul 09 04:12:13.271390 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:13.271301 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f866bfd4d-hcqsn" event={"ID":"cb2d01ed-df03-41ea-8ca1-2e85d9231e07","Type":"ContainerStarted","Data":"c46f2154bf4613fdd0f560dddd52c594b7c0cbc96b6966d65e250211787a49bd"} Jul 09 04:12:13.309494 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:13.309437 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f866bfd4d-hcqsn" podStartSLOduration=1.309418562 podStartE2EDuration="1.309418562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:12:13.30926075 +0000 UTC m=+290.546230929" watchObservedRunningTime="2026-07-09 04:12:13.309418562 +0000 UTC m=+290.546388741" Jul 09 04:12:19.226460 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:12:19.226419 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vqlpv" podUID="65b7ad76-b3e5-48b9-abcc-a556fbc1e905" Jul 09 04:12:19.486473 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:19.486370 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:21.266096 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:21.266066 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2q2pz" Jul 09 04:12:22.681361 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:22.681321 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:22.681781 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:22.681386 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:22.686371 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:22.686349 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:23.115656 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:23.115577 2584 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 04:12:23.312903 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:23.312746 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:12:23.361692 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:23.361662 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8cb5498d7-2h52f"] Jul 09 04:12:30.268537 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:30.268491 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2q2pz_66216ebf-6785-49d8-acf3-ba7de5e1ef2c/dns/0.log" Jul 09 04:12:30.274247 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:30.274227 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2q2pz_66216ebf-6785-49d8-acf3-ba7de5e1ef2c/kube-rbac-proxy/0.log" Jul 09 04:12:30.654433 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:30.654407 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bcknn_e5450f12-4d48-4380-b616-623f5b43eaa6/dns-node-resolver/0.log" Jul 09 04:12:32.225623 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:32.225586 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:12:34.165350 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.165286 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76c76dd6b7-7mx4q" podUID="5e0b086b-d51b-452a-aa81-9d62ec6feaa9" containerName="console" containerID="cri-o://ef916ce9a0ff958b0ffbba368b087f669d074022e0715194a94568f9a7f1874d" gracePeriod=15 Jul 09 04:12:34.350807 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.350777 2584 generic.go:356] "Generic (PLEG): container finished" podID="5e0b086b-d51b-452a-aa81-9d62ec6feaa9" containerID="ef916ce9a0ff958b0ffbba368b087f669d074022e0715194a94568f9a7f1874d" exitCode=0 Jul 09 04:12:34.350947 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.350822 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76c76dd6b7-7mx4q" event={"ID":"5e0b086b-d51b-452a-aa81-9d62ec6feaa9","Type":"ContainerDied","Data":"ef916ce9a0ff958b0ffbba368b087f669d074022e0715194a94568f9a7f1874d"} Jul 09 04:12:34.404176 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.404154 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:34.423180 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423092 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-trusted-ca-bundle\") pod \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " Jul 09 04:12:34.423180 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423124 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-config\") pod \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " Jul 09 04:12:34.423392 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423188 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-service-ca\") pod \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " Jul 09 04:12:34.423392 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423354 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-serving-cert\") pod \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " Jul 09 04:12:34.423505 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423417 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-oauth-serving-cert\") pod \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " Jul 09 04:12:34.423505 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423449 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jwmx\" (UniqueName: \"kubernetes.io/projected/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-kube-api-access-7jwmx\") pod \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " Jul 09 04:12:34.423505 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423480 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-oauth-config\") pod \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\" (UID: \"5e0b086b-d51b-452a-aa81-9d62ec6feaa9\") " Jul 09 04:12:34.423680 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423558 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-service-ca" (OuterVolumeSpecName: "service-ca") pod "5e0b086b-d51b-452a-aa81-9d62ec6feaa9" (UID: "5e0b086b-d51b-452a-aa81-9d62ec6feaa9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:34.423680 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423571 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-config" (OuterVolumeSpecName: "console-config") pod "5e0b086b-d51b-452a-aa81-9d62ec6feaa9" (UID: "5e0b086b-d51b-452a-aa81-9d62ec6feaa9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:34.423884 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423835 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:34.423884 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423862 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:34.424081 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.423901 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5e0b086b-d51b-452a-aa81-9d62ec6feaa9" (UID: "5e0b086b-d51b-452a-aa81-9d62ec6feaa9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:34.424792 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.424755 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5e0b086b-d51b-452a-aa81-9d62ec6feaa9" (UID: "5e0b086b-d51b-452a-aa81-9d62ec6feaa9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:34.426560 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.426513 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-kube-api-access-7jwmx" (OuterVolumeSpecName: "kube-api-access-7jwmx") pod "5e0b086b-d51b-452a-aa81-9d62ec6feaa9" (UID: "5e0b086b-d51b-452a-aa81-9d62ec6feaa9"). InnerVolumeSpecName "kube-api-access-7jwmx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:34.426717 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.426698 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5e0b086b-d51b-452a-aa81-9d62ec6feaa9" (UID: "5e0b086b-d51b-452a-aa81-9d62ec6feaa9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:34.426779 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.426728 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5e0b086b-d51b-452a-aa81-9d62ec6feaa9" (UID: "5e0b086b-d51b-452a-aa81-9d62ec6feaa9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:34.524353 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.524320 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:34.524353 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.524350 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-oauth-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:34.524353 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.524361 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jwmx\" (UniqueName: \"kubernetes.io/projected/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-kube-api-access-7jwmx\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:34.524630 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.524371 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-console-oauth-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:34.524630 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:34.524379 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b086b-d51b-452a-aa81-9d62ec6feaa9-trusted-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:35.356031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:35.356002 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76c76dd6b7-7mx4q" Jul 09 04:12:35.356031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:35.356010 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76c76dd6b7-7mx4q" event={"ID":"5e0b086b-d51b-452a-aa81-9d62ec6feaa9","Type":"ContainerDied","Data":"3cb2f21a96f99478d53bced2daf789e09419d7b615d07fd04fae05e1550028b9"} Jul 09 04:12:35.356540 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:35.356066 2584 scope.go:117] "RemoveContainer" containerID="ef916ce9a0ff958b0ffbba368b087f669d074022e0715194a94568f9a7f1874d" Jul 09 04:12:35.371267 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:35.371239 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76c76dd6b7-7mx4q"] Jul 09 04:12:35.374424 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:35.374402 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76c76dd6b7-7mx4q"] Jul 09 04:12:37.229583 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.229553 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e0b086b-d51b-452a-aa81-9d62ec6feaa9" path="/var/lib/kubelet/pods/5e0b086b-d51b-452a-aa81-9d62ec6feaa9/volumes" Jul 09 04:12:37.333041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.333004 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5df89db6f9-mlv5r" podUID="28de9a4a-db2b-4b7f-a0e1-95dece215053" containerName="console" containerID="cri-o://b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8" gracePeriod=15 Jul 09 04:12:37.579131 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.579107 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:37.646836 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.646800 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-oauth-serving-cert\") pod \"28de9a4a-db2b-4b7f-a0e1-95dece215053\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " Jul 09 04:12:37.647031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.646850 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-oauth-config\") pod \"28de9a4a-db2b-4b7f-a0e1-95dece215053\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " Jul 09 04:12:37.647031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.646878 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-service-ca\") pod \"28de9a4a-db2b-4b7f-a0e1-95dece215053\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " Jul 09 04:12:37.647031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.646905 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-trusted-ca-bundle\") pod \"28de9a4a-db2b-4b7f-a0e1-95dece215053\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " Jul 09 04:12:37.647182 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647034 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-config\") pod \"28de9a4a-db2b-4b7f-a0e1-95dece215053\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " Jul 09 04:12:37.647182 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647094 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-serving-cert\") pod \"28de9a4a-db2b-4b7f-a0e1-95dece215053\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " Jul 09 04:12:37.647182 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647132 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crv7f\" (UniqueName: \"kubernetes.io/projected/28de9a4a-db2b-4b7f-a0e1-95dece215053-kube-api-access-crv7f\") pod \"28de9a4a-db2b-4b7f-a0e1-95dece215053\" (UID: \"28de9a4a-db2b-4b7f-a0e1-95dece215053\") " Jul 09 04:12:37.647329 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647264 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "28de9a4a-db2b-4b7f-a0e1-95dece215053" (UID: "28de9a4a-db2b-4b7f-a0e1-95dece215053"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:37.647384 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647357 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "28de9a4a-db2b-4b7f-a0e1-95dece215053" (UID: "28de9a4a-db2b-4b7f-a0e1-95dece215053"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:37.647384 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647369 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-service-ca" (OuterVolumeSpecName: "service-ca") pod "28de9a4a-db2b-4b7f-a0e1-95dece215053" (UID: "28de9a4a-db2b-4b7f-a0e1-95dece215053"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:37.647471 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647412 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-oauth-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:37.647471 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.647453 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-config" (OuterVolumeSpecName: "console-config") pod "28de9a4a-db2b-4b7f-a0e1-95dece215053" (UID: "28de9a4a-db2b-4b7f-a0e1-95dece215053"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:37.649386 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.649359 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "28de9a4a-db2b-4b7f-a0e1-95dece215053" (UID: "28de9a4a-db2b-4b7f-a0e1-95dece215053"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:37.649500 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.649385 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "28de9a4a-db2b-4b7f-a0e1-95dece215053" (UID: "28de9a4a-db2b-4b7f-a0e1-95dece215053"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:37.649500 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.649439 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28de9a4a-db2b-4b7f-a0e1-95dece215053-kube-api-access-crv7f" (OuterVolumeSpecName: "kube-api-access-crv7f") pod "28de9a4a-db2b-4b7f-a0e1-95dece215053" (UID: "28de9a4a-db2b-4b7f-a0e1-95dece215053"). InnerVolumeSpecName "kube-api-access-crv7f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:37.748676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.748567 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-oauth-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:37.748676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.748598 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:37.748676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.748613 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-trusted-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:37.748676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.748624 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:37.748676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.748637 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/28de9a4a-db2b-4b7f-a0e1-95dece215053-console-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:37.748676 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:37.748660 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crv7f\" (UniqueName: \"kubernetes.io/projected/28de9a4a-db2b-4b7f-a0e1-95dece215053-kube-api-access-crv7f\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:38.373258 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.373217 2584 generic.go:356] "Generic (PLEG): container finished" podID="28de9a4a-db2b-4b7f-a0e1-95dece215053" containerID="b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8" exitCode=0 Jul 09 04:12:38.373694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.373286 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5df89db6f9-mlv5r" Jul 09 04:12:38.373694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.373301 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5df89db6f9-mlv5r" event={"ID":"28de9a4a-db2b-4b7f-a0e1-95dece215053","Type":"ContainerDied","Data":"b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8"} Jul 09 04:12:38.373694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.373385 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5df89db6f9-mlv5r" event={"ID":"28de9a4a-db2b-4b7f-a0e1-95dece215053","Type":"ContainerDied","Data":"5f7df5ebc75ea3aec7c2243a512f75155ec52918353c999686d859557275489c"} Jul 09 04:12:38.373694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.373409 2584 scope.go:117] "RemoveContainer" containerID="b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8" Jul 09 04:12:38.382187 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.382168 2584 scope.go:117] "RemoveContainer" containerID="b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8" Jul 09 04:12:38.382471 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:12:38.382453 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8\": container with ID starting with b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8 not found: ID does not exist" containerID="b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8" Jul 09 04:12:38.382515 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.382479 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8"} err="failed to get container status \"b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8\": rpc error: code = NotFound desc = could not find container \"b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8\": container with ID starting with b442918a171674cabc13f935263cb46c16849b38e623ea79f09652758e7d75d8 not found: ID does not exist" Jul 09 04:12:38.393312 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.393277 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5df89db6f9-mlv5r"] Jul 09 04:12:38.399297 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:38.399276 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5df89db6f9-mlv5r"] Jul 09 04:12:39.230367 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:39.230335 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28de9a4a-db2b-4b7f-a0e1-95dece215053" path="/var/lib/kubelet/pods/28de9a4a-db2b-4b7f-a0e1-95dece215053/volumes" Jul 09 04:12:48.382164 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.382097 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8cb5498d7-2h52f" podUID="46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" containerName="console" containerID="cri-o://e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699" gracePeriod=15 Jul 09 04:12:48.625695 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.625668 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:12:48.749386 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749354 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-oauth-config\") pod \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " Jul 09 04:12:48.749602 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749403 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6km9r\" (UniqueName: \"kubernetes.io/projected/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-kube-api-access-6km9r\") pod \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " Jul 09 04:12:48.749602 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749433 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-config\") pod \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " Jul 09 04:12:48.749602 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749458 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-service-ca\") pod \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " Jul 09 04:12:48.749602 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749495 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-serving-cert\") pod \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " Jul 09 04:12:48.749602 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749513 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-oauth-serving-cert\") pod \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\" (UID: \"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb\") " Jul 09 04:12:48.750013 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749878 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-config" (OuterVolumeSpecName: "console-config") pod "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" (UID: "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:48.750013 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.749988 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-service-ca" (OuterVolumeSpecName: "service-ca") pod "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" (UID: "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:48.750179 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.750019 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" (UID: "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:12:48.752015 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.751991 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" (UID: "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:48.752254 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.752238 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-kube-api-access-6km9r" (OuterVolumeSpecName: "kube-api-access-6km9r") pod "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" (UID: "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb"). InnerVolumeSpecName "kube-api-access-6km9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:12:48.752309 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.752290 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" (UID: "46ffe3b2-07c8-43cc-9a4f-c755246e5dfb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:12:48.850573 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.850510 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6km9r\" (UniqueName: \"kubernetes.io/projected/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-kube-api-access-6km9r\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:48.850573 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.850561 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:48.850573 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.850570 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:48.850573 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.850579 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:48.850573 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.850588 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-oauth-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:48.850875 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:48.850596 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb-console-oauth-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:12:49.417016 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.416984 2584 generic.go:356] "Generic (PLEG): container finished" podID="46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" containerID="e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699" exitCode=0 Jul 09 04:12:49.417450 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.417030 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cb5498d7-2h52f" Jul 09 04:12:49.417450 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.417064 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cb5498d7-2h52f" event={"ID":"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb","Type":"ContainerDied","Data":"e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699"} Jul 09 04:12:49.417450 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.417103 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cb5498d7-2h52f" event={"ID":"46ffe3b2-07c8-43cc-9a4f-c755246e5dfb","Type":"ContainerDied","Data":"5f736873ae13866327b75e33cdff2984a18911561c2f91454866bf1660eca96e"} Jul 09 04:12:49.417450 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.417123 2584 scope.go:117] "RemoveContainer" containerID="e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699" Jul 09 04:12:49.425347 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.425325 2584 scope.go:117] "RemoveContainer" containerID="e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699" Jul 09 04:12:49.425710 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:12:49.425689 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699\": container with ID starting with e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699 not found: ID does not exist" containerID="e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699" Jul 09 04:12:49.425789 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.425717 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699"} err="failed to get container status \"e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699\": rpc error: code = NotFound desc = could not find container \"e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699\": container with ID starting with e50127cb7d9fb0f0c07f95c17936f78ae55f6ab276b89c946ea4652a0e916699 not found: ID does not exist" Jul 09 04:12:49.431932 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.431911 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8cb5498d7-2h52f"] Jul 09 04:12:49.435238 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:49.435216 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8cb5498d7-2h52f"] Jul 09 04:12:51.230145 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:12:51.230115 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" path="/var/lib/kubelet/pods/46ffe3b2-07c8-43cc-9a4f-c755246e5dfb/volumes" Jul 09 04:13:22.628493 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:22.628406 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:13:22.629041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:22.628734 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="alertmanager" containerID="cri-o://b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423" gracePeriod=120 Jul 09 04:13:22.629041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:22.628804 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-metric" containerID="cri-o://bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a" gracePeriod=120 Jul 09 04:13:22.629041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:22.628805 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-web" containerID="cri-o://7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0" gracePeriod=120 Jul 09 04:13:22.629041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:22.628877 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="config-reloader" containerID="cri-o://5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87" gracePeriod=120 Jul 09 04:13:22.629041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:22.628899 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy" containerID="cri-o://c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5" gracePeriod=120 Jul 09 04:13:22.629041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:22.628855 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="prom-label-proxy" containerID="cri-o://9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8" gracePeriod=120 Jul 09 04:13:23.543708 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543642 2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerID="9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8" exitCode=0 Jul 09 04:13:23.543708 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543701 2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerID="c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5" exitCode=0 Jul 09 04:13:23.543708 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543710 2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerID="5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87" exitCode=0 Jul 09 04:13:23.543708 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543715 2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerID="b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423" exitCode=0 Jul 09 04:13:23.544005 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543755 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8"} Jul 09 04:13:23.544005 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543821 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5"} Jul 09 04:13:23.544005 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543848 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87"} Jul 09 04:13:23.544005 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.543874 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423"} Jul 09 04:13:23.883269 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.883246 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:23.949882 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.949849 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-metrics-client-ca\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950068 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.949892 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-tls-assets\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950068 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.949918 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-cluster-tls-config\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950068 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.949955 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950068 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950007 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-out\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950068 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950033 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-web-config\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950068 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950057 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-main-db\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950352 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950087 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-volume\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950352 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950115 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-trusted-ca-bundle\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950352 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950144 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c8skc\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-kube-api-access-c8skc\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950352 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950177 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950352 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950208 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-web\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950352 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950237 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls\") pod \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\" (UID: \"9b0d4951-d154-431c-adb6-db1a18ec8b8e\") " Jul 09 04:13:23.950352 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950287 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:23.950698 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950483 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-metrics-client-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:23.950757 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.950730 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:13:23.951321 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.951079 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:13:23.953987 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.953515 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:23.953987 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.953541 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:13:23.953987 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.953655 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-volume" (OuterVolumeSpecName: "config-volume") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:23.953987 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.953678 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:23.954448 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.954423 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-kube-api-access-c8skc" (OuterVolumeSpecName: "kube-api-access-c8skc") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "kube-api-access-c8skc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:13:23.955919 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.955884 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-out" (OuterVolumeSpecName: "config-out") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 04:13:23.956033 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.956005 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:23.956332 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.956309 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:23.958076 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.958056 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:23.963516 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:23.963493 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-web-config" (OuterVolumeSpecName: "web-config") pod "9b0d4951-d154-431c-adb6-db1a18ec8b8e" (UID: "9b0d4951-d154-431c-adb6-db1a18ec8b8e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051260 2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-out\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051302 2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-web-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051312 2584 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-main-db\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051323 2584 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-config-volume\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051333 2584 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9b0d4951-d154-431c-adb6-db1a18ec8b8e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051342 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c8skc\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-kube-api-access-c8skc\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051351 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051359 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051369 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-main-tls\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051363 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051378 2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9b0d4951-d154-431c-adb6-db1a18ec8b8e-tls-assets\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051817 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051386 2584 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-cluster-tls-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.051817 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.051394 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9b0d4951-d154-431c-adb6-db1a18ec8b8e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:13:24.551375 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.551341 2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerID="bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a" exitCode=0 Jul 09 04:13:24.551375 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.551367 2584 generic.go:356] "Generic (PLEG): container finished" podID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerID="7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0" exitCode=0 Jul 09 04:13:24.551639 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.551423 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a"} Jul 09 04:13:24.551639 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.551439 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.551639 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.551497 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0"} Jul 09 04:13:24.551639 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.551550 2584 scope.go:117] "RemoveContainer" containerID="9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8" Jul 09 04:13:24.551639 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.551557 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9b0d4951-d154-431c-adb6-db1a18ec8b8e","Type":"ContainerDied","Data":"d2be8f2b09a000c2f1a6e3d457ce4281091e480a8629b6dd64d03880bfe8a116"} Jul 09 04:13:24.560375 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.560355 2584 scope.go:117] "RemoveContainer" containerID="bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a" Jul 09 04:13:24.568373 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.568352 2584 scope.go:117] "RemoveContainer" containerID="c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5" Jul 09 04:13:24.574082 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.574052 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:13:24.576425 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.576404 2584 scope.go:117] "RemoveContainer" containerID="7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0" Jul 09 04:13:24.581961 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.579483 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:13:24.586798 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.586772 2584 scope.go:117] "RemoveContainer" containerID="5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87" Jul 09 04:13:24.595152 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.595126 2584 scope.go:117] "RemoveContainer" containerID="b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423" Jul 09 04:13:24.602297 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602261 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:13:24.602569 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602555 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="init-config-reloader" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602571 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="init-config-reloader" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602580 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="config-reloader" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602585 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="config-reloader" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602593 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-web" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602598 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-web" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602606 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" containerName="console" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602611 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" containerName="console" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602617 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0b086b-d51b-452a-aa81-9d62ec6feaa9" containerName="console" Jul 09 04:13:24.602624 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602622 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0b086b-d51b-452a-aa81-9d62ec6feaa9" containerName="console" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602631 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-metric" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602636 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-metric" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602644 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602649 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602656 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28de9a4a-db2b-4b7f-a0e1-95dece215053" containerName="console" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602661 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="28de9a4a-db2b-4b7f-a0e1-95dece215053" containerName="console" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602667 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="prom-label-proxy" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602671 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="prom-label-proxy" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602679 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="alertmanager" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602684 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="alertmanager" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602726 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-metric" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602733 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="alertmanager" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602739 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy-web" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602744 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="kube-rbac-proxy" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602750 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e0b086b-d51b-452a-aa81-9d62ec6feaa9" containerName="console" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602754 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="28de9a4a-db2b-4b7f-a0e1-95dece215053" containerName="console" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602760 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="config-reloader" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602765 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="46ffe3b2-07c8-43cc-9a4f-c755246e5dfb" containerName="console" Jul 09 04:13:24.602880 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.602770 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" containerName="prom-label-proxy" Jul 09 04:13:24.604020 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.604002 2584 scope.go:117] "RemoveContainer" containerID="dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357" Jul 09 04:13:24.606128 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.606108 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.608622 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.608595 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 09 04:13:24.608812 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.608772 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 09 04:13:24.608907 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.608780 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 09 04:13:24.608952 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.608917 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 09 04:13:24.608990 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.608952 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-sgz4j\"" Jul 09 04:13:24.609034 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.609008 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 09 04:13:24.609092 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.609078 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 09 04:13:24.609162 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.609133 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 09 04:13:24.609219 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.609186 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 09 04:13:24.613260 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.613236 2584 scope.go:117] "RemoveContainer" containerID="9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8" Jul 09 04:13:24.613607 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:13:24.613590 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8\": container with ID starting with 9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8 not found: ID does not exist" containerID="9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8" Jul 09 04:13:24.613683 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.613615 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8"} err="failed to get container status \"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8\": rpc error: code = NotFound desc = could not find container \"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8\": container with ID starting with 9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8 not found: ID does not exist" Jul 09 04:13:24.613683 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.613633 2584 scope.go:117] "RemoveContainer" containerID="bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a" Jul 09 04:13:24.613872 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:13:24.613851 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a\": container with ID starting with bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a not found: ID does not exist" containerID="bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a" Jul 09 04:13:24.613942 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.613881 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a"} err="failed to get container status \"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a\": rpc error: code = NotFound desc = could not find container \"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a\": container with ID starting with bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a not found: ID does not exist" Jul 09 04:13:24.613942 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.613904 2584 scope.go:117] "RemoveContainer" containerID="c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5" Jul 09 04:13:24.614046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.614027 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 09 04:13:24.614236 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:13:24.614213 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5\": container with ID starting with c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5 not found: ID does not exist" containerID="c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5" Jul 09 04:13:24.614286 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.614243 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5"} err="failed to get container status \"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5\": rpc error: code = NotFound desc = could not find container \"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5\": container with ID starting with c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5 not found: ID does not exist" Jul 09 04:13:24.614286 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.614259 2584 scope.go:117] "RemoveContainer" containerID="7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0" Jul 09 04:13:24.614661 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:13:24.614484 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0\": container with ID starting with 7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0 not found: ID does not exist" containerID="7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0" Jul 09 04:13:24.614661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.614510 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0"} err="failed to get container status \"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0\": rpc error: code = NotFound desc = could not find container \"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0\": container with ID starting with 7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0 not found: ID does not exist" Jul 09 04:13:24.614661 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.614555 2584 scope.go:117] "RemoveContainer" containerID="5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87" Jul 09 04:13:24.614920 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:13:24.614898 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87\": container with ID starting with 5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87 not found: ID does not exist" containerID="5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87" Jul 09 04:13:24.614999 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.614926 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87"} err="failed to get container status \"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87\": rpc error: code = NotFound desc = could not find container \"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87\": container with ID starting with 5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87 not found: ID does not exist" Jul 09 04:13:24.614999 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.614941 2584 scope.go:117] "RemoveContainer" containerID="b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423" Jul 09 04:13:24.615189 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:13:24.615172 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423\": container with ID starting with b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423 not found: ID does not exist" containerID="b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423" Jul 09 04:13:24.615246 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615190 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423"} err="failed to get container status \"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423\": rpc error: code = NotFound desc = could not find container \"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423\": container with ID starting with b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423 not found: ID does not exist" Jul 09 04:13:24.615246 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615204 2584 scope.go:117] "RemoveContainer" containerID="dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357" Jul 09 04:13:24.615428 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:13:24.615412 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357\": container with ID starting with dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357 not found: ID does not exist" containerID="dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357" Jul 09 04:13:24.615469 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615432 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357"} err="failed to get container status \"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357\": rpc error: code = NotFound desc = could not find container \"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357\": container with ID starting with dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357 not found: ID does not exist" Jul 09 04:13:24.615469 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615446 2584 scope.go:117] "RemoveContainer" containerID="9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8" Jul 09 04:13:24.615662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615646 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8"} err="failed to get container status \"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8\": rpc error: code = NotFound desc = could not find container \"9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8\": container with ID starting with 9ef67dd2949cd5f6e76f024a996ae9c30c81087a12d5d5a0311a45a45869cbb8 not found: ID does not exist" Jul 09 04:13:24.615711 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615661 2584 scope.go:117] "RemoveContainer" containerID="bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a" Jul 09 04:13:24.615910 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615887 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a"} err="failed to get container status \"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a\": rpc error: code = NotFound desc = could not find container \"bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a\": container with ID starting with bf43bf6987a734d1e06c367837e174ac7bb50ad7cbf534b8b3c37eb45d26289a not found: ID does not exist" Jul 09 04:13:24.615985 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.615913 2584 scope.go:117] "RemoveContainer" containerID="c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5" Jul 09 04:13:24.616108 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616092 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5"} err="failed to get container status \"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5\": rpc error: code = NotFound desc = could not find container \"c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5\": container with ID starting with c5f438cbb86d6430a5f848dd686d927dfdf2ba24d62880c623ce377a1485eab5 not found: ID does not exist" Jul 09 04:13:24.616147 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616108 2584 scope.go:117] "RemoveContainer" containerID="7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0" Jul 09 04:13:24.616298 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616280 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0"} err="failed to get container status \"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0\": rpc error: code = NotFound desc = could not find container \"7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0\": container with ID starting with 7fc166e79df1d90aebb56a09ea7c891d5b1066b80266e8063957650f83f96ea0 not found: ID does not exist" Jul 09 04:13:24.616298 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616295 2584 scope.go:117] "RemoveContainer" containerID="5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87" Jul 09 04:13:24.616532 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616500 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87"} err="failed to get container status \"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87\": rpc error: code = NotFound desc = could not find container \"5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87\": container with ID starting with 5f42f397f716dee64622e8df46a858cc441aab8269ec5fe133b912a31fe37c87 not found: ID does not exist" Jul 09 04:13:24.616590 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616537 2584 scope.go:117] "RemoveContainer" containerID="b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423" Jul 09 04:13:24.616787 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616766 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423"} err="failed to get container status \"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423\": rpc error: code = NotFound desc = could not find container \"b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423\": container with ID starting with b6d1194e5cb4b602768c36a08cb9ef246140c0882a2b3df7df89db4214519423 not found: ID does not exist" Jul 09 04:13:24.616864 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616788 2584 scope.go:117] "RemoveContainer" containerID="dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357" Jul 09 04:13:24.617008 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.616988 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357"} err="failed to get container status \"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357\": rpc error: code = NotFound desc = could not find container \"dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357\": container with ID starting with dd35781e44ff14ed1c59cc25ec1fc03b8bda9d4c7b04b344e734bb9d351b2357 not found: ID does not exist" Jul 09 04:13:24.619890 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.619870 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:13:24.655487 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655444 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655502 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655559 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-web-config\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655582 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/60667dfa-bb62-44e4-b457-fd5f194b5958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60667dfa-bb62-44e4-b457-fd5f194b5958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655624 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60667dfa-bb62-44e4-b457-fd5f194b5958-config-out\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655669 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655869 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655715 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655869 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655743 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655869 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655760 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60667dfa-bb62-44e4-b457-fd5f194b5958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655869 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655788 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-config-volume\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655869 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655821 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9t6w4\" (UniqueName: \"kubernetes.io/projected/60667dfa-bb62-44e4-b457-fd5f194b5958-kube-api-access-9t6w4\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.655869 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.655839 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60667dfa-bb62-44e4-b457-fd5f194b5958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.756983 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.756938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60667dfa-bb62-44e4-b457-fd5f194b5958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.756983 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.756990 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757177 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757116 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757177 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757151 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-web-config\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757263 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757176 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/60667dfa-bb62-44e4-b457-fd5f194b5958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757263 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60667dfa-bb62-44e4-b457-fd5f194b5958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757263 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757212 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60667dfa-bb62-44e4-b457-fd5f194b5958-config-out\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757263 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757246 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757303 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757347 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757372 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60667dfa-bb62-44e4-b457-fd5f194b5958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757404 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-config-volume\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757436 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9t6w4\" (UniqueName: \"kubernetes.io/projected/60667dfa-bb62-44e4-b457-fd5f194b5958-kube-api-access-9t6w4\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757770 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757617 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/60667dfa-bb62-44e4-b457-fd5f194b5958-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.757970 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.757947 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60667dfa-bb62-44e4-b457-fd5f194b5958-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.758830 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.758803 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60667dfa-bb62-44e4-b457-fd5f194b5958-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.760541 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.760497 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-web-config\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.760883 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.760853 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.760982 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.760893 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.760982 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.760970 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/60667dfa-bb62-44e4-b457-fd5f194b5958-tls-assets\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.761093 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.761002 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.761093 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.761062 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/60667dfa-bb62-44e4-b457-fd5f194b5958-config-out\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.761321 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.761301 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.761904 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.761883 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-config-volume\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.762784 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.762762 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/60667dfa-bb62-44e4-b457-fd5f194b5958-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.765045 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.765024 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9t6w4\" (UniqueName: \"kubernetes.io/projected/60667dfa-bb62-44e4-b457-fd5f194b5958-kube-api-access-9t6w4\") pod \"alertmanager-main-0\" (UID: \"60667dfa-bb62-44e4-b457-fd5f194b5958\") " pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:24.917809 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:24.917763 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 09 04:13:25.065220 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:25.065038 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 09 04:13:25.067440 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:13:25.067409 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60667dfa_bb62_44e4_b457_fd5f194b5958.slice/crio-290e797f3986ebb07c36e306ac4ae7233df8738754e06c9211d9f48e99d06e3c WatchSource:0}: Error finding container 290e797f3986ebb07c36e306ac4ae7233df8738754e06c9211d9f48e99d06e3c: Status 404 returned error can't find the container with id 290e797f3986ebb07c36e306ac4ae7233df8738754e06c9211d9f48e99d06e3c Jul 09 04:13:25.229878 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:25.229807 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b0d4951-d154-431c-adb6-db1a18ec8b8e" path="/var/lib/kubelet/pods/9b0d4951-d154-431c-adb6-db1a18ec8b8e/volumes" Jul 09 04:13:25.557481 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:25.557394 2584 generic.go:356] "Generic (PLEG): container finished" podID="60667dfa-bb62-44e4-b457-fd5f194b5958" containerID="ce32bafb96099ccb16c77bb28ba6025d48d0567044374fc3ecd2ff432109712e" exitCode=0 Jul 09 04:13:25.557481 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:25.557434 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerDied","Data":"ce32bafb96099ccb16c77bb28ba6025d48d0567044374fc3ecd2ff432109712e"} Jul 09 04:13:25.557663 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:25.557494 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerStarted","Data":"290e797f3986ebb07c36e306ac4ae7233df8738754e06c9211d9f48e99d06e3c"} Jul 09 04:13:26.563707 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:26.563673 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerStarted","Data":"ea28495a2e034176f11bda24085dfa8cf529fc2249a38aeb5c7b287ce291b647"} Jul 09 04:13:26.564092 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:26.563739 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerStarted","Data":"b1b8574acfd5a99f19ebf0dd50b8e5f587fc27bd3151a4fd12f57b8528c13de8"} Jul 09 04:13:26.564092 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:26.563765 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerStarted","Data":"50762ef52f21582c6debfe2f8306909a7ff5f4af33e781f2c46d7d2d81c85e2d"} Jul 09 04:13:26.564092 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:26.563788 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerStarted","Data":"6851faefd31c8db2e47660c00939b4045a82bad643c059b087b71bfa3f2905ff"} Jul 09 04:13:26.564092 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:26.563814 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerStarted","Data":"590e642040bdc01a59b6d74ef9edd1c173a4aea86ef319831f5f471d6a22aff6"} Jul 09 04:13:26.564092 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:26.563836 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"60667dfa-bb62-44e4-b457-fd5f194b5958","Type":"ContainerStarted","Data":"ee39b1f2b634f9f303655310903d7c0846d8837aeebf1942691e8bf170e1a2cb"} Jul 09 04:13:26.589459 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:26.588965 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.57456597 podStartE2EDuration="2.588943572s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.377602ms" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:26.587253694 +0000 UTC m=+363.824223874" watchObservedRunningTime="2026-07-09 04:13:26.588943572 +0000 UTC m=+363.825913753" Jul 09 04:13:27.692693 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.692652 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-857f44b44f-rwjr4"] Jul 09 04:13:27.694820 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.694797 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.709063 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.709024 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f44b44f-rwjr4"] Jul 09 04:13:27.784954 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.784922 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-config\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.785123 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.784963 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-service-ca\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.785123 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.784994 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-serving-cert\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.785123 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.785068 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-oauth-config\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.785123 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.785098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-oauth-serving-cert\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.785265 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.785157 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-trusted-ca-bundle\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.785265 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.785188 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88gxd\" (UniqueName: \"kubernetes.io/projected/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-kube-api-access-88gxd\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.886533 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.886491 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-trusted-ca-bundle\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.886694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.886556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88gxd\" (UniqueName: \"kubernetes.io/projected/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-kube-api-access-88gxd\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.886694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.886625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-config\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.886694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.886653 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-service-ca\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.886694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.886670 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-serving-cert\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.886905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.886699 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-oauth-config\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.886905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.886721 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-oauth-serving-cert\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.887449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.887390 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-config\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.887449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.887437 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-service-ca\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.887449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.887450 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-oauth-serving-cert\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.887449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.887476 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-trusted-ca-bundle\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.889298 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.889265 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-oauth-config\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.889421 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.889402 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-serving-cert\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:27.894674 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:27.894648 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88gxd\" (UniqueName: \"kubernetes.io/projected/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-kube-api-access-88gxd\") pod \"console-857f44b44f-rwjr4\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:28.003449 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:28.003363 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:28.139589 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:28.139468 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-857f44b44f-rwjr4"] Jul 09 04:13:28.141831 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:13:28.141805 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15059af9_a69f_4880_b5a8_1d1a3fd12ee2.slice/crio-e63e25aac08f141e46926cb55e9374f79d0c9d80efad503c02256dacaef2f65e WatchSource:0}: Error finding container e63e25aac08f141e46926cb55e9374f79d0c9d80efad503c02256dacaef2f65e: Status 404 returned error can't find the container with id e63e25aac08f141e46926cb55e9374f79d0c9d80efad503c02256dacaef2f65e Jul 09 04:13:28.572052 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:28.572006 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f44b44f-rwjr4" event={"ID":"15059af9-a69f-4880-b5a8-1d1a3fd12ee2","Type":"ContainerStarted","Data":"33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b"} Jul 09 04:13:28.572203 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:28.572067 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f44b44f-rwjr4" event={"ID":"15059af9-a69f-4880-b5a8-1d1a3fd12ee2","Type":"ContainerStarted","Data":"e63e25aac08f141e46926cb55e9374f79d0c9d80efad503c02256dacaef2f65e"} Jul 09 04:13:28.589441 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:28.589387 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-857f44b44f-rwjr4" podStartSLOduration=1.5893692640000001 podStartE2EDuration="1.589369264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:13:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:28.588362252 +0000 UTC m=+365.825332450" watchObservedRunningTime="2026-07-09 04:13:28.589369264 +0000 UTC m=+365.826339442" Jul 09 04:13:36.148454 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:36.148413 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:13:36.151012 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:36.150986 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/65b7ad76-b3e5-48b9-abcc-a556fbc1e905-metrics-certs\") pod \"network-metrics-daemon-vqlpv\" (UID: \"65b7ad76-b3e5-48b9-abcc-a556fbc1e905\") " pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:13:36.428626 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:36.428547 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v2z2w\"" Jul 09 04:13:36.436775 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:36.436745 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vqlpv" Jul 09 04:13:36.561781 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:36.561740 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vqlpv"] Jul 09 04:13:36.571279 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:36.571245 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:13:36.603339 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:36.603305 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqlpv" event={"ID":"65b7ad76-b3e5-48b9-abcc-a556fbc1e905","Type":"ContainerStarted","Data":"c08aecb6fbe7c18af245785d2242fc2fde4afc0efed944b37b1d59e8081ed7c3"} Jul 09 04:13:37.609209 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:37.609179 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqlpv" event={"ID":"65b7ad76-b3e5-48b9-abcc-a556fbc1e905","Type":"ContainerStarted","Data":"49280f2f2daf9c77abcd2441853f2073664657ca0c860d45d2345633f5b115fa"} Jul 09 04:13:38.004566 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:38.004484 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:38.004761 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:38.004592 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:38.009485 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:38.009456 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:38.615134 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:38.615084 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vqlpv" event={"ID":"65b7ad76-b3e5-48b9-abcc-a556fbc1e905","Type":"ContainerStarted","Data":"8e95fa39f8fa0e39f3d8682ebde3ef3262132375261769e36ab1a6658ab07810"} Jul 09 04:13:38.619094 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:38.619070 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:13:38.631189 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:38.631130 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vqlpv" podStartSLOduration=374.689361268 podStartE2EDuration="6m15.631115895s" totalImagesPullingTime="941.754627ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:13:38.630192966 +0000 UTC m=+375.867163144" watchObservedRunningTime="2026-07-09 04:13:38.631115895 +0000 UTC m=+375.868086072" Jul 09 04:13:38.678219 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:13:38.678175 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f866bfd4d-hcqsn"] Jul 09 04:14:03.700470 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:03.700405 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f866bfd4d-hcqsn" podUID="cb2d01ed-df03-41ea-8ca1-2e85d9231e07" containerName="console" containerID="cri-o://ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75" gracePeriod=15 Jul 09 04:14:03.950132 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:03.950107 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:14:04.075506 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075404 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-trusted-ca-bundle\") pod \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " Jul 09 04:14:04.075506 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075470 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-oauth-serving-cert\") pod \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " Jul 09 04:14:04.075776 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075510 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mr2dr\" (UniqueName: \"kubernetes.io/projected/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-kube-api-access-mr2dr\") pod \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " Jul 09 04:14:04.075776 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075565 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-config\") pod \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " Jul 09 04:14:04.075776 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075708 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-serving-cert\") pod \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " Jul 09 04:14:04.075776 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075753 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-service-ca\") pod \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " Jul 09 04:14:04.075985 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075807 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-oauth-config\") pod \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\" (UID: \"cb2d01ed-df03-41ea-8ca1-2e85d9231e07\") " Jul 09 04:14:04.075985 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075904 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cb2d01ed-df03-41ea-8ca1-2e85d9231e07" (UID: "cb2d01ed-df03-41ea-8ca1-2e85d9231e07"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:14:04.076095 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.075976 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb2d01ed-df03-41ea-8ca1-2e85d9231e07" (UID: "cb2d01ed-df03-41ea-8ca1-2e85d9231e07"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:14:04.076146 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.076103 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-config" (OuterVolumeSpecName: "console-config") pod "cb2d01ed-df03-41ea-8ca1-2e85d9231e07" (UID: "cb2d01ed-df03-41ea-8ca1-2e85d9231e07"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:14:04.076200 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.076172 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb2d01ed-df03-41ea-8ca1-2e85d9231e07" (UID: "cb2d01ed-df03-41ea-8ca1-2e85d9231e07"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:14:04.076241 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.076195 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-trusted-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:14:04.076241 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.076214 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-oauth-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:14:04.076241 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.076229 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:14:04.077902 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.077881 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-kube-api-access-mr2dr" (OuterVolumeSpecName: "kube-api-access-mr2dr") pod "cb2d01ed-df03-41ea-8ca1-2e85d9231e07" (UID: "cb2d01ed-df03-41ea-8ca1-2e85d9231e07"). InnerVolumeSpecName "kube-api-access-mr2dr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:14:04.077902 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.077889 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb2d01ed-df03-41ea-8ca1-2e85d9231e07" (UID: "cb2d01ed-df03-41ea-8ca1-2e85d9231e07"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:14:04.077998 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.077962 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb2d01ed-df03-41ea-8ca1-2e85d9231e07" (UID: "cb2d01ed-df03-41ea-8ca1-2e85d9231e07"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:14:04.176662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.176621 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-oauth-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:14:04.176662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.176650 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mr2dr\" (UniqueName: \"kubernetes.io/projected/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-kube-api-access-mr2dr\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:14:04.176662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.176660 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-console-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:14:04.176662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.176677 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb2d01ed-df03-41ea-8ca1-2e85d9231e07-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:14:04.710589 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.710555 2584 generic.go:356] "Generic (PLEG): container finished" podID="cb2d01ed-df03-41ea-8ca1-2e85d9231e07" containerID="ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75" exitCode=0 Jul 09 04:14:04.711030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.710609 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f866bfd4d-hcqsn" event={"ID":"cb2d01ed-df03-41ea-8ca1-2e85d9231e07","Type":"ContainerDied","Data":"ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75"} Jul 09 04:14:04.711030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.710652 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f866bfd4d-hcqsn" Jul 09 04:14:04.711030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.710661 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f866bfd4d-hcqsn" event={"ID":"cb2d01ed-df03-41ea-8ca1-2e85d9231e07","Type":"ContainerDied","Data":"c46f2154bf4613fdd0f560dddd52c594b7c0cbc96b6966d65e250211787a49bd"} Jul 09 04:14:04.711030 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.710694 2584 scope.go:117] "RemoveContainer" containerID="ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75" Jul 09 04:14:04.719474 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.719454 2584 scope.go:117] "RemoveContainer" containerID="ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75" Jul 09 04:14:04.719775 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:14:04.719756 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75\": container with ID starting with ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75 not found: ID does not exist" containerID="ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75" Jul 09 04:14:04.719846 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.719782 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75"} err="failed to get container status \"ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75\": rpc error: code = NotFound desc = could not find container \"ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75\": container with ID starting with ca01e7dc2b7ca85d36d0a2cbe282affc9f637c3e0783e7e74ff282923e98ce75 not found: ID does not exist" Jul 09 04:14:04.729700 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.729675 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f866bfd4d-hcqsn"] Jul 09 04:14:04.731958 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:04.731935 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f866bfd4d-hcqsn"] Jul 09 04:14:05.229459 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:14:05.229427 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb2d01ed-df03-41ea-8ca1-2e85d9231e07" path="/var/lib/kubelet/pods/cb2d01ed-df03-41ea-8ca1-2e85d9231e07/volumes" Jul 09 04:15:06.308066 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.308030 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wvwzm"] Jul 09 04:15:06.308589 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.308330 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb2d01ed-df03-41ea-8ca1-2e85d9231e07" containerName="console" Jul 09 04:15:06.308589 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.308347 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb2d01ed-df03-41ea-8ca1-2e85d9231e07" containerName="console" Jul 09 04:15:06.308589 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.308442 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb2d01ed-df03-41ea-8ca1-2e85d9231e07" containerName="console" Jul 09 04:15:06.311279 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.311263 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.313702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.313683 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 04:15:06.313782 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.313724 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 04:15:06.315133 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.315095 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-q4rqz\"" Jul 09 04:15:06.323442 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.323414 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wvwzm"] Jul 09 04:15:06.343021 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.342985 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/249d3916-1787-4699-a324-284045d75f0f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wvwzm\" (UID: \"249d3916-1787-4699-a324-284045d75f0f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.343199 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.343120 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kfkm\" (UniqueName: \"kubernetes.io/projected/249d3916-1787-4699-a324-284045d75f0f-kube-api-access-2kfkm\") pod \"cert-manager-webhook-6486ffd997-wvwzm\" (UID: \"249d3916-1787-4699-a324-284045d75f0f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.444256 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.444216 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kfkm\" (UniqueName: \"kubernetes.io/projected/249d3916-1787-4699-a324-284045d75f0f-kube-api-access-2kfkm\") pod \"cert-manager-webhook-6486ffd997-wvwzm\" (UID: \"249d3916-1787-4699-a324-284045d75f0f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.444422 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.444278 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/249d3916-1787-4699-a324-284045d75f0f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wvwzm\" (UID: \"249d3916-1787-4699-a324-284045d75f0f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.452390 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.452362 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/249d3916-1787-4699-a324-284045d75f0f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wvwzm\" (UID: \"249d3916-1787-4699-a324-284045d75f0f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.452511 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.452434 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kfkm\" (UniqueName: \"kubernetes.io/projected/249d3916-1787-4699-a324-284045d75f0f-kube-api-access-2kfkm\") pod \"cert-manager-webhook-6486ffd997-wvwzm\" (UID: \"249d3916-1787-4699-a324-284045d75f0f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.623589 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.623464 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:06.757844 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.757814 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wvwzm"] Jul 09 04:15:06.760214 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:15:06.760179 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod249d3916_1787_4699_a324_284045d75f0f.slice/crio-48a99b157e8668382782d2a73ca14237c4b09d063e3c013cfe9b40f8cbafd439 WatchSource:0}: Error finding container 48a99b157e8668382782d2a73ca14237c4b09d063e3c013cfe9b40f8cbafd439: Status 404 returned error can't find the container with id 48a99b157e8668382782d2a73ca14237c4b09d063e3c013cfe9b40f8cbafd439 Jul 09 04:15:06.934821 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:06.934786 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" event={"ID":"249d3916-1787-4699-a324-284045d75f0f","Type":"ContainerStarted","Data":"48a99b157e8668382782d2a73ca14237c4b09d063e3c013cfe9b40f8cbafd439"} Jul 09 04:15:10.951660 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:10.951624 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" event={"ID":"249d3916-1787-4699-a324-284045d75f0f","Type":"ContainerStarted","Data":"c243ea4c5fd954d97787440ede6de0f409fa4478bf162be6cf37f65e9a178567"} Jul 09 04:15:10.952104 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:10.951756 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:10.986003 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:10.985927 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" podStartSLOduration=1.220468539 podStartE2EDuration="4.985912377s" totalImagesPullingTime="3.765443838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:10.984566242 +0000 UTC m=+468.221536418" watchObservedRunningTime="2026-07-09 04:15:10.985912377 +0000 UTC m=+468.222882554" Jul 09 04:15:16.957883 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:16.957848 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wvwzm" Jul 09 04:15:18.197509 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.197471 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-554d2"] Jul 09 04:15:18.212475 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.212446 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-554d2"] Jul 09 04:15:18.212840 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.212823 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.217494 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.217471 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-rns2h\"" Jul 09 04:15:18.346607 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.346569 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a01450ba-4502-401a-9447-f6a74f35cb0b-bound-sa-token\") pod \"cert-manager-55b4d954d5-554d2\" (UID: \"a01450ba-4502-401a-9447-f6a74f35cb0b\") " pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.346607 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.346610 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75pwl\" (UniqueName: \"kubernetes.io/projected/a01450ba-4502-401a-9447-f6a74f35cb0b-kube-api-access-75pwl\") pod \"cert-manager-55b4d954d5-554d2\" (UID: \"a01450ba-4502-401a-9447-f6a74f35cb0b\") " pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.447914 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.447809 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a01450ba-4502-401a-9447-f6a74f35cb0b-bound-sa-token\") pod \"cert-manager-55b4d954d5-554d2\" (UID: \"a01450ba-4502-401a-9447-f6a74f35cb0b\") " pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.447914 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.447872 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75pwl\" (UniqueName: \"kubernetes.io/projected/a01450ba-4502-401a-9447-f6a74f35cb0b-kube-api-access-75pwl\") pod \"cert-manager-55b4d954d5-554d2\" (UID: \"a01450ba-4502-401a-9447-f6a74f35cb0b\") " pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.457137 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.457108 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a01450ba-4502-401a-9447-f6a74f35cb0b-bound-sa-token\") pod \"cert-manager-55b4d954d5-554d2\" (UID: \"a01450ba-4502-401a-9447-f6a74f35cb0b\") " pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.457244 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.457226 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75pwl\" (UniqueName: \"kubernetes.io/projected/a01450ba-4502-401a-9447-f6a74f35cb0b-kube-api-access-75pwl\") pod \"cert-manager-55b4d954d5-554d2\" (UID: \"a01450ba-4502-401a-9447-f6a74f35cb0b\") " pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.527183 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.527141 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-554d2" Jul 09 04:15:18.662582 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.662542 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-554d2"] Jul 09 04:15:18.665342 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:15:18.665311 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda01450ba_4502_401a_9447_f6a74f35cb0b.slice/crio-cf16d48b02c303983fdf17c868c005204c67e791a010da83d186bea7ec23efe1 WatchSource:0}: Error finding container cf16d48b02c303983fdf17c868c005204c67e791a010da83d186bea7ec23efe1: Status 404 returned error can't find the container with id cf16d48b02c303983fdf17c868c005204c67e791a010da83d186bea7ec23efe1 Jul 09 04:15:18.987514 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.987421 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-554d2" event={"ID":"a01450ba-4502-401a-9447-f6a74f35cb0b","Type":"ContainerStarted","Data":"0bc38cf545aec08d9db5dc08a6cd78c255fa7c8a7383c007976a19970939c0a3"} Jul 09 04:15:18.987514 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:18.987459 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-554d2" event={"ID":"a01450ba-4502-401a-9447-f6a74f35cb0b","Type":"ContainerStarted","Data":"cf16d48b02c303983fdf17c868c005204c67e791a010da83d186bea7ec23efe1"} Jul 09 04:15:19.002184 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:19.002133 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-554d2" podStartSLOduration=1.002116876 podStartE2EDuration="1.002116876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:19.001013241 +0000 UTC m=+476.237983435" watchObservedRunningTime="2026-07-09 04:15:19.002116876 +0000 UTC m=+476.239087052" Jul 09 04:15:37.731386 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.731356 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn"] Jul 09 04:15:37.734612 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.734592 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:37.737366 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.737344 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4xqf2\"" Jul 09 04:15:37.737845 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.737827 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 04:15:37.737963 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.737847 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 04:15:37.737963 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.737858 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 04:15:37.737963 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.737846 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 04:15:37.751192 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.751163 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn"] Jul 09 04:15:37.911811 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.911777 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d02ffa15-e098-4367-92fd-0e0c9167df22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:37.911811 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.911811 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jlnq\" (UniqueName: \"kubernetes.io/projected/d02ffa15-e098-4367-92fd-0e0c9167df22-kube-api-access-5jlnq\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:37.912012 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.911864 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d02ffa15-e098-4367-92fd-0e0c9167df22-manifests\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:37.912012 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.911901 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d02ffa15-e098-4367-92fd-0e0c9167df22-tmp\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:37.912012 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:37.911920 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d02ffa15-e098-4367-92fd-0e0c9167df22-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.012771 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.012691 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d02ffa15-e098-4367-92fd-0e0c9167df22-manifests\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.012771 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.012730 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d02ffa15-e098-4367-92fd-0e0c9167df22-tmp\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.012771 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.012763 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d02ffa15-e098-4367-92fd-0e0c9167df22-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.013034 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.012802 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d02ffa15-e098-4367-92fd-0e0c9167df22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.013034 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.012827 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jlnq\" (UniqueName: \"kubernetes.io/projected/d02ffa15-e098-4367-92fd-0e0c9167df22-kube-api-access-5jlnq\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.013144 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.013124 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d02ffa15-e098-4367-92fd-0e0c9167df22-manifests\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.015157 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.015132 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d02ffa15-e098-4367-92fd-0e0c9167df22-tmp\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.015263 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.015230 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d02ffa15-e098-4367-92fd-0e0c9167df22-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.015460 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.015438 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d02ffa15-e098-4367-92fd-0e0c9167df22-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.022313 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.022287 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jlnq\" (UniqueName: \"kubernetes.io/projected/d02ffa15-e098-4367-92fd-0e0c9167df22-kube-api-access-5jlnq\") pod \"opendatahub-operator-controller-manager-5cff547d94-wfxxn\" (UID: \"d02ffa15-e098-4367-92fd-0e0c9167df22\") " pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.045251 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.045224 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:38.186336 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:38.186304 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn"] Jul 09 04:15:38.189645 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:15:38.189612 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd02ffa15_e098_4367_92fd_0e0c9167df22.slice/crio-6cb3f1e054ec132458731efc55ff8312946d9791a25a1d5d3443e169b8352ee7 WatchSource:0}: Error finding container 6cb3f1e054ec132458731efc55ff8312946d9791a25a1d5d3443e169b8352ee7: Status 404 returned error can't find the container with id 6cb3f1e054ec132458731efc55ff8312946d9791a25a1d5d3443e169b8352ee7 Jul 09 04:15:39.064222 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:39.064174 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" event={"ID":"d02ffa15-e098-4367-92fd-0e0c9167df22","Type":"ContainerStarted","Data":"6cb3f1e054ec132458731efc55ff8312946d9791a25a1d5d3443e169b8352ee7"} Jul 09 04:15:42.082065 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:42.082024 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" event={"ID":"d02ffa15-e098-4367-92fd-0e0c9167df22","Type":"ContainerStarted","Data":"d015cf8a031f2b74de8fc86b7b8ea81af80f65aa5578ee3be9e801f135837ebf"} Jul 09 04:15:43.087697 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:43.087664 2584 generic.go:356] "Generic (PLEG): container finished" podID="d02ffa15-e098-4367-92fd-0e0c9167df22" containerID="d015cf8a031f2b74de8fc86b7b8ea81af80f65aa5578ee3be9e801f135837ebf" exitCode=0 Jul 09 04:15:43.088184 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:43.087700 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" event={"ID":"d02ffa15-e098-4367-92fd-0e0c9167df22","Type":"ContainerDied","Data":"d015cf8a031f2b74de8fc86b7b8ea81af80f65aa5578ee3be9e801f135837ebf"} Jul 09 04:15:44.093311 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:44.093274 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" event={"ID":"d02ffa15-e098-4367-92fd-0e0c9167df22","Type":"ContainerStarted","Data":"35c82ea3d8fb848e009e327f8a6d5adcece80baabc0a68d8609f3b8b73583586"} Jul 09 04:15:44.093791 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:44.093349 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:44.118049 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:44.117992 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" podStartSLOduration=2.468085209 podStartE2EDuration="7.117976028s" totalImagesPullingTime="3.590594449s" totalInitContainerRuntime="1.05929637s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:15:44.115677528 +0000 UTC m=+501.352647709" watchObservedRunningTime="2026-07-09 04:15:44.117976028 +0000 UTC m=+501.354946206" Jul 09 04:15:55.099421 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:55.099392 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cff547d94-wfxxn" Jul 09 04:15:57.939822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:57.939790 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j8cdg"] Jul 09 04:15:57.946836 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:57.946817 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:57.952067 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:57.951999 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d74231e1-0159-4ab1-8d87-6dee366ea477-cert\") pod \"odh-model-controller-7998c67dc4-j8cdg\" (UID: \"d74231e1-0159-4ab1-8d87-6dee366ea477\") " pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:57.952298 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:57.952270 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zwnm\" (UniqueName: \"kubernetes.io/projected/d74231e1-0159-4ab1-8d87-6dee366ea477-kube-api-access-6zwnm\") pod \"odh-model-controller-7998c67dc4-j8cdg\" (UID: \"d74231e1-0159-4ab1-8d87-6dee366ea477\") " pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:57.954084 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:57.954066 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 09 04:15:57.954184 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:57.954081 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-vck7z\"" Jul 09 04:15:57.968540 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:57.968493 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j8cdg"] Jul 09 04:15:58.052906 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.052873 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d74231e1-0159-4ab1-8d87-6dee366ea477-cert\") pod \"odh-model-controller-7998c67dc4-j8cdg\" (UID: \"d74231e1-0159-4ab1-8d87-6dee366ea477\") " pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:58.053099 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.052914 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zwnm\" (UniqueName: \"kubernetes.io/projected/d74231e1-0159-4ab1-8d87-6dee366ea477-kube-api-access-6zwnm\") pod \"odh-model-controller-7998c67dc4-j8cdg\" (UID: \"d74231e1-0159-4ab1-8d87-6dee366ea477\") " pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:58.053099 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:15:58.053028 2584 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 09 04:15:58.053184 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:15:58.053103 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d74231e1-0159-4ab1-8d87-6dee366ea477-cert podName:d74231e1-0159-4ab1-8d87-6dee366ea477 nodeName:}" failed. No retries permitted until 2026-07-09 04:15:58.553084934 +0000 UTC m=+515.790055091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d74231e1-0159-4ab1-8d87-6dee366ea477-cert") pod "odh-model-controller-7998c67dc4-j8cdg" (UID: "d74231e1-0159-4ab1-8d87-6dee366ea477") : secret "odh-model-controller-webhook-cert" not found Jul 09 04:15:58.071006 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.070970 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zwnm\" (UniqueName: \"kubernetes.io/projected/d74231e1-0159-4ab1-8d87-6dee366ea477-kube-api-access-6zwnm\") pod \"odh-model-controller-7998c67dc4-j8cdg\" (UID: \"d74231e1-0159-4ab1-8d87-6dee366ea477\") " pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:58.556695 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.556651 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d74231e1-0159-4ab1-8d87-6dee366ea477-cert\") pod \"odh-model-controller-7998c67dc4-j8cdg\" (UID: \"d74231e1-0159-4ab1-8d87-6dee366ea477\") " pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:58.559365 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.559336 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d74231e1-0159-4ab1-8d87-6dee366ea477-cert\") pod \"odh-model-controller-7998c67dc4-j8cdg\" (UID: \"d74231e1-0159-4ab1-8d87-6dee366ea477\") " pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:58.565620 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.565590 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p"] Jul 09 04:15:58.569059 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.569028 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.572042 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.572014 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dzm27\"" Jul 09 04:15:58.572431 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.572408 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 04:15:58.572599 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.572558 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 04:15:58.572798 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.572784 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 04:15:58.573144 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.573124 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 04:15:58.583021 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.582992 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p"] Jul 09 04:15:58.657855 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.657814 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c025e8a3-7bea-428c-b592-85a126570924-tls-certs\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.658050 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.657879 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c025e8a3-7bea-428c-b592-85a126570924-tmp\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.658050 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.657897 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h964\" (UniqueName: \"kubernetes.io/projected/c025e8a3-7bea-428c-b592-85a126570924-kube-api-access-7h964\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.719269 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.719228 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fhmnf"] Jul 09 04:15:58.722419 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.722393 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:58.725420 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.725386 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xg4vk\"" Jul 09 04:15:58.725564 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.725482 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 04:15:58.733646 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.733621 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fhmnf"] Jul 09 04:15:58.758247 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.758205 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c025e8a3-7bea-428c-b592-85a126570924-tmp\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.758247 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.758246 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h964\" (UniqueName: \"kubernetes.io/projected/c025e8a3-7bea-428c-b592-85a126570924-kube-api-access-7h964\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.758501 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.758372 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3545f05-462f-4a55-8fba-01b632f438a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-fhmnf\" (UID: \"a3545f05-462f-4a55-8fba-01b632f438a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:58.758501 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.758414 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx5cr\" (UniqueName: \"kubernetes.io/projected/a3545f05-462f-4a55-8fba-01b632f438a2-kube-api-access-wx5cr\") pod \"model-serving-api-54d89bd5fc-fhmnf\" (UID: \"a3545f05-462f-4a55-8fba-01b632f438a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:58.758635 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.758538 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c025e8a3-7bea-428c-b592-85a126570924-tls-certs\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.760726 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.760699 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c025e8a3-7bea-428c-b592-85a126570924-tmp\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.761124 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.761102 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c025e8a3-7bea-428c-b592-85a126570924-tls-certs\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.767666 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.767638 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h964\" (UniqueName: \"kubernetes.io/projected/c025e8a3-7bea-428c-b592-85a126570924-kube-api-access-7h964\") pod \"kube-auth-proxy-58ffb9d954-z4j2p\" (UID: \"c025e8a3-7bea-428c-b592-85a126570924\") " pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:58.857394 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.857303 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:15:58.859654 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.859625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3545f05-462f-4a55-8fba-01b632f438a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-fhmnf\" (UID: \"a3545f05-462f-4a55-8fba-01b632f438a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:58.859775 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.859670 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx5cr\" (UniqueName: \"kubernetes.io/projected/a3545f05-462f-4a55-8fba-01b632f438a2-kube-api-access-wx5cr\") pod \"model-serving-api-54d89bd5fc-fhmnf\" (UID: \"a3545f05-462f-4a55-8fba-01b632f438a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:58.859862 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:15:58.859837 2584 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 09 04:15:58.859949 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:15:58.859936 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3545f05-462f-4a55-8fba-01b632f438a2-tls-certs podName:a3545f05-462f-4a55-8fba-01b632f438a2 nodeName:}" failed. No retries permitted until 2026-07-09 04:15:59.359913551 +0000 UTC m=+516.596883708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a3545f05-462f-4a55-8fba-01b632f438a2-tls-certs") pod "model-serving-api-54d89bd5fc-fhmnf" (UID: "a3545f05-462f-4a55-8fba-01b632f438a2") : secret "model-serving-api-tls" not found Jul 09 04:15:58.869191 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.869160 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx5cr\" (UniqueName: \"kubernetes.io/projected/a3545f05-462f-4a55-8fba-01b632f438a2-kube-api-access-wx5cr\") pod \"model-serving-api-54d89bd5fc-fhmnf\" (UID: \"a3545f05-462f-4a55-8fba-01b632f438a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:58.884782 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:58.884746 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" Jul 09 04:15:59.021201 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.021172 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-j8cdg"] Jul 09 04:15:59.023417 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:15:59.023390 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74231e1_0159_4ab1_8d87_6dee366ea477.slice/crio-c5fd60ee0c98a88b1081843d6607034696de63302683fc18367a56d06b1d4713 WatchSource:0}: Error finding container c5fd60ee0c98a88b1081843d6607034696de63302683fc18367a56d06b1d4713: Status 404 returned error can't find the container with id c5fd60ee0c98a88b1081843d6607034696de63302683fc18367a56d06b1d4713 Jul 09 04:15:59.072701 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.072669 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p"] Jul 09 04:15:59.075624 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:15:59.075597 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc025e8a3_7bea_428c_b592_85a126570924.slice/crio-8a27227757ebd8370acd11b2c5a6ba35105ff38a2db4acff3d8175866b8f9060 WatchSource:0}: Error finding container 8a27227757ebd8370acd11b2c5a6ba35105ff38a2db4acff3d8175866b8f9060: Status 404 returned error can't find the container with id 8a27227757ebd8370acd11b2c5a6ba35105ff38a2db4acff3d8175866b8f9060 Jul 09 04:15:59.151951 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.151913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" event={"ID":"d74231e1-0159-4ab1-8d87-6dee366ea477","Type":"ContainerStarted","Data":"c5fd60ee0c98a88b1081843d6607034696de63302683fc18367a56d06b1d4713"} Jul 09 04:15:59.152985 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.152958 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" event={"ID":"c025e8a3-7bea-428c-b592-85a126570924","Type":"ContainerStarted","Data":"8a27227757ebd8370acd11b2c5a6ba35105ff38a2db4acff3d8175866b8f9060"} Jul 09 04:15:59.364378 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.364340 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3545f05-462f-4a55-8fba-01b632f438a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-fhmnf\" (UID: \"a3545f05-462f-4a55-8fba-01b632f438a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:59.366971 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.366941 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3545f05-462f-4a55-8fba-01b632f438a2-tls-certs\") pod \"model-serving-api-54d89bd5fc-fhmnf\" (UID: \"a3545f05-462f-4a55-8fba-01b632f438a2\") " pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:59.633698 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.633670 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:15:59.764633 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:15:59.764559 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fhmnf"] Jul 09 04:15:59.767179 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:15:59.767149 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3545f05_462f_4a55_8fba_01b632f438a2.slice/crio-d28038500072f7f8e182966c42f0ad83eb14fca8826f1084860f90da065f5de3 WatchSource:0}: Error finding container d28038500072f7f8e182966c42f0ad83eb14fca8826f1084860f90da065f5de3: Status 404 returned error can't find the container with id d28038500072f7f8e182966c42f0ad83eb14fca8826f1084860f90da065f5de3 Jul 09 04:16:00.159292 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:00.159252 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" event={"ID":"a3545f05-462f-4a55-8fba-01b632f438a2","Type":"ContainerStarted","Data":"d28038500072f7f8e182966c42f0ad83eb14fca8826f1084860f90da065f5de3"} Jul 09 04:16:04.178438 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.178400 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" event={"ID":"d74231e1-0159-4ab1-8d87-6dee366ea477","Type":"ContainerStarted","Data":"27b472bcc0fd0833f2150c90b866d71be13aaeaa4f07e27189aba11f547777a7"} Jul 09 04:16:04.178959 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.178636 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:16:04.179972 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.179943 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" event={"ID":"c025e8a3-7bea-428c-b592-85a126570924","Type":"ContainerStarted","Data":"e6b3c6eaaa54ee2984885a991de0ffe10576d871155d954c129d5a6304856229"} Jul 09 04:16:04.181451 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.181424 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" event={"ID":"a3545f05-462f-4a55-8fba-01b632f438a2","Type":"ContainerStarted","Data":"79d23c2f121377ea783a4f5f684ed44f49f56adae2dfb06e349b616f4f17ec67"} Jul 09 04:16:04.181573 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.181561 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:16:04.203631 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.203571 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" podStartSLOduration=2.891746404 podStartE2EDuration="7.203512134s" totalImagesPullingTime="4.31176573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:04.203062312 +0000 UTC m=+521.440032497" watchObservedRunningTime="2026-07-09 04:16:04.203512134 +0000 UTC m=+521.440482312" Jul 09 04:16:04.224746 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.224682 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" podStartSLOduration=2.657171331 podStartE2EDuration="6.224661308s" totalImagesPullingTime="3.567489977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:04.224043195 +0000 UTC m=+521.461013377" watchObservedRunningTime="2026-07-09 04:16:04.224661308 +0000 UTC m=+521.461631489" Jul 09 04:16:04.654740 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.654686 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-58ffb9d954-z4j2p" podStartSLOduration=2.388889637 podStartE2EDuration="6.654664086s" totalImagesPullingTime="4.265774449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:04.242043406 +0000 UTC m=+521.479013606" watchObservedRunningTime="2026-07-09 04:16:04.654664086 +0000 UTC m=+521.891634265" Jul 09 04:16:04.655874 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.655851 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm"] Jul 09 04:16:04.659269 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.659250 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.662052 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.662024 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 09 04:16:04.662177 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.662125 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-sfmq7\"" Jul 09 04:16:04.689391 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.687139 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm"] Jul 09 04:16:04.705736 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705699 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.705736 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705735 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.705934 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.705934 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705822 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.705934 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705853 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.705934 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705871 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.705934 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705902 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.705934 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.706160 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.705936 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cbhz\" (UniqueName: \"kubernetes.io/projected/e357a8ee-eb6e-45bb-80a6-c0aa15966536-kube-api-access-4cbhz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.806917 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.806862 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807156 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.806961 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807156 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.806991 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807156 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807021 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807156 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807078 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807156 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807106 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807156 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807136 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cbhz\" (UniqueName: \"kubernetes.io/projected/e357a8ee-eb6e-45bb-80a6-c0aa15966536-kube-api-access-4cbhz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807552 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807234 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807552 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807371 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807552 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807260 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807729 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807683 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807729 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807690 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.807833 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.807749 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.808298 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.808277 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.810342 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.810314 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.810553 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.810511 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.819150 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.819125 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cbhz\" (UniqueName: \"kubernetes.io/projected/e357a8ee-eb6e-45bb-80a6-c0aa15966536-kube-api-access-4cbhz\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.819461 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.819439 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e357a8ee-eb6e-45bb-80a6-c0aa15966536-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf48c4dm\" (UID: \"e357a8ee-eb6e-45bb-80a6-c0aa15966536\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:04.969335 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:04.969241 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:05.106373 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:05.106342 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm"] Jul 09 04:16:05.109471 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:16:05.109442 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode357a8ee_eb6e_45bb_80a6_c0aa15966536.slice/crio-22b2852a4081b664ddbfd1fd0f1e2914d1373752d775681866c9e58512aab9ad WatchSource:0}: Error finding container 22b2852a4081b664ddbfd1fd0f1e2914d1373752d775681866c9e58512aab9ad: Status 404 returned error can't find the container with id 22b2852a4081b664ddbfd1fd0f1e2914d1373752d775681866c9e58512aab9ad Jul 09 04:16:05.187210 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:05.187176 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" event={"ID":"e357a8ee-eb6e-45bb-80a6-c0aa15966536","Type":"ContainerStarted","Data":"22b2852a4081b664ddbfd1fd0f1e2914d1373752d775681866c9e58512aab9ad"} Jul 09 04:16:07.637093 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.637058 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj"] Jul 09 04:16:07.649213 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.649179 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.653489 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.653459 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 04:16:07.654026 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.653999 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 04:16:07.654905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.654779 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 04:16:07.654905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.654829 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 04:16:07.654905 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.654838 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bst2k\"" Jul 09 04:16:07.655822 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.655804 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 04:16:07.682555 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.682496 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj"] Jul 09 04:16:07.738211 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.738180 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6f1474e8-f5e7-4f46-b332-a640d86fc6af-manager-config\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.738431 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.738226 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f1474e8-f5e7-4f46-b332-a640d86fc6af-cert\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.738431 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.738349 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mncr4\" (UniqueName: \"kubernetes.io/projected/6f1474e8-f5e7-4f46-b332-a640d86fc6af-kube-api-access-mncr4\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.738431 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.738388 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f1474e8-f5e7-4f46-b332-a640d86fc6af-metrics-cert\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.838888 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.838853 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6f1474e8-f5e7-4f46-b332-a640d86fc6af-manager-config\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.838888 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.838894 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f1474e8-f5e7-4f46-b332-a640d86fc6af-cert\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.838888 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.838945 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mncr4\" (UniqueName: \"kubernetes.io/projected/6f1474e8-f5e7-4f46-b332-a640d86fc6af-kube-api-access-mncr4\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.839242 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.838964 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f1474e8-f5e7-4f46-b332-a640d86fc6af-metrics-cert\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.839918 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.839777 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6f1474e8-f5e7-4f46-b332-a640d86fc6af-manager-config\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.841854 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.841834 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6f1474e8-f5e7-4f46-b332-a640d86fc6af-metrics-cert\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.842142 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.842119 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f1474e8-f5e7-4f46-b332-a640d86fc6af-cert\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.857669 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.857637 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mncr4\" (UniqueName: \"kubernetes.io/projected/6f1474e8-f5e7-4f46-b332-a640d86fc6af-kube-api-access-mncr4\") pod \"lws-controller-manager-645bc6d555-6fzcj\" (UID: \"6f1474e8-f5e7-4f46-b332-a640d86fc6af\") " pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:07.923603 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.923561 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:07.923677 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.923655 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:07.923723 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.923692 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 09 04:16:07.962001 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:07.961970 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:08.115692 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:08.115497 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj"] Jul 09 04:16:08.118183 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:16:08.118149 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1474e8_f5e7_4f46_b332_a640d86fc6af.slice/crio-1ae100a1dad8bd2c2b090f57a85e2db88578fb71a2e4f311bf37fb95d923bc58 WatchSource:0}: Error finding container 1ae100a1dad8bd2c2b090f57a85e2db88578fb71a2e4f311bf37fb95d923bc58: Status 404 returned error can't find the container with id 1ae100a1dad8bd2c2b090f57a85e2db88578fb71a2e4f311bf37fb95d923bc58 Jul 09 04:16:08.203153 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:08.203046 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" event={"ID":"e357a8ee-eb6e-45bb-80a6-c0aa15966536","Type":"ContainerStarted","Data":"5b55a2fa0fe6afacad926328eb334900a47bcfd163ea094a7aca3efe05d22a25"} Jul 09 04:16:08.204293 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:08.204265 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" event={"ID":"6f1474e8-f5e7-4f46-b332-a640d86fc6af","Type":"ContainerStarted","Data":"1ae100a1dad8bd2c2b090f57a85e2db88578fb71a2e4f311bf37fb95d923bc58"} Jul 09 04:16:08.245267 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:08.245216 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" podStartSLOduration=1.4334535910000001 podStartE2EDuration="4.245195477s" totalImagesPullingTime="2.811741886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:16:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:08.243560976 +0000 UTC m=+525.480531156" watchObservedRunningTime="2026-07-09 04:16:08.245195477 +0000 UTC m=+525.482165656" Jul 09 04:16:08.970136 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:08.970099 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:09.975097 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:09.975067 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:10.216799 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:10.216770 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:10.217692 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:10.217669 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf48c4dm" Jul 09 04:16:11.220990 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:11.220949 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" event={"ID":"6f1474e8-f5e7-4f46-b332-a640d86fc6af","Type":"ContainerStarted","Data":"c89aa1d5982d0a932a8b2e440b6538f1d303dec0b6234c3331c62c1c7c846b53"} Jul 09 04:16:11.236949 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:11.236901 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" podStartSLOduration=1.633718117 podStartE2EDuration="4.236887542s" totalImagesPullingTime="2.603169425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:16:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:16:11.235401876 +0000 UTC m=+528.472372072" watchObservedRunningTime="2026-07-09 04:16:11.236887542 +0000 UTC m=+528.473857720" Jul 09 04:16:12.225888 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:12.225855 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:16:15.189890 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:15.189810 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-j8cdg" Jul 09 04:16:15.191689 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:15.191667 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fhmnf" Jul 09 04:16:23.232958 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:16:23.232931 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-645bc6d555-6fzcj" Jul 09 04:17:26.370766 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.370729 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c54c65b84-fpnpf"] Jul 09 04:17:26.374386 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.374369 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.386754 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.386724 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c54c65b84-fpnpf"] Jul 09 04:17:26.453427 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.453391 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-serving-cert\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.453427 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.453429 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-oauth-config\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.453720 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.453465 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-config\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.453720 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.453515 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-trusted-ca-bundle\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.453720 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.453592 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-654g7\" (UniqueName: \"kubernetes.io/projected/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-kube-api-access-654g7\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.453720 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.453619 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-oauth-serving-cert\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.453720 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.453643 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-service-ca\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.554670 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.554632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-654g7\" (UniqueName: \"kubernetes.io/projected/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-kube-api-access-654g7\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.554670 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.554671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-oauth-serving-cert\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.554929 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.554690 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-service-ca\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.554929 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.554721 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-serving-cert\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.554929 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.554736 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-oauth-config\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.554929 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.554764 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-config\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.554929 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.554830 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-trusted-ca-bundle\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.555569 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.555511 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-oauth-serving-cert\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.555687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.555585 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-config\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.555687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.555673 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-trusted-ca-bundle\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.555765 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.555744 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-service-ca\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.557371 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.557349 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-oauth-config\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.557424 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.557379 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-console-serving-cert\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.562538 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.562499 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-654g7\" (UniqueName: \"kubernetes.io/projected/a8de99d4-df02-4c55-85c3-d3b5ea2e00bf-kube-api-access-654g7\") pod \"console-c54c65b84-fpnpf\" (UID: \"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf\") " pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.683706 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.683665 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:26.820910 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:26.820875 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c54c65b84-fpnpf"] Jul 09 04:17:26.824045 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:17:26.824004 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8de99d4_df02_4c55_85c3_d3b5ea2e00bf.slice/crio-0d708837ad608a396ff3ebc68281991c319732b3b88e62c9079be1f8663c57e4 WatchSource:0}: Error finding container 0d708837ad608a396ff3ebc68281991c319732b3b88e62c9079be1f8663c57e4: Status 404 returned error can't find the container with id 0d708837ad608a396ff3ebc68281991c319732b3b88e62c9079be1f8663c57e4 Jul 09 04:17:27.551507 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:27.551469 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c54c65b84-fpnpf" event={"ID":"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf","Type":"ContainerStarted","Data":"a6571bba81475b0a351149448f3ab18dc7796b98364e64982c43d93579fe9b5e"} Jul 09 04:17:27.551926 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:27.551550 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c54c65b84-fpnpf" event={"ID":"a8de99d4-df02-4c55-85c3-d3b5ea2e00bf","Type":"ContainerStarted","Data":"0d708837ad608a396ff3ebc68281991c319732b3b88e62c9079be1f8663c57e4"} Jul 09 04:17:27.572196 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:27.572139 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c54c65b84-fpnpf" podStartSLOduration=1.572122757 podStartE2EDuration="1.572122757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:17:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:17:27.570590219 +0000 UTC m=+604.807560395" watchObservedRunningTime="2026-07-09 04:17:27.572122757 +0000 UTC m=+604.809093000" Jul 09 04:17:36.683845 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:36.683799 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:36.684294 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:36.683914 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:36.688746 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:36.688721 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:37.596570 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:37.596543 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c54c65b84-fpnpf" Jul 09 04:17:37.651011 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:17:37.650978 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857f44b44f-rwjr4"] Jul 09 04:18:02.671824 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.671769 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-857f44b44f-rwjr4" podUID="15059af9-a69f-4880-b5a8-1d1a3fd12ee2" containerName="console" containerID="cri-o://33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b" gracePeriod=15 Jul 09 04:18:02.917738 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.917711 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:18:02.986851 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.986754 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-service-ca\") pod \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " Jul 09 04:18:02.986851 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.986810 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-oauth-config\") pod \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " Jul 09 04:18:02.986851 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.986846 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-config\") pod \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " Jul 09 04:18:02.987130 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.986868 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-serving-cert\") pod \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " Jul 09 04:18:02.987130 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.986905 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-trusted-ca-bundle\") pod \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " Jul 09 04:18:02.987130 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.987006 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88gxd\" (UniqueName: \"kubernetes.io/projected/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-kube-api-access-88gxd\") pod \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " Jul 09 04:18:02.987130 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.987051 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-oauth-serving-cert\") pod \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\" (UID: \"15059af9-a69f-4880-b5a8-1d1a3fd12ee2\") " Jul 09 04:18:02.987304 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.987200 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-service-ca" (OuterVolumeSpecName: "service-ca") pod "15059af9-a69f-4880-b5a8-1d1a3fd12ee2" (UID: "15059af9-a69f-4880-b5a8-1d1a3fd12ee2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:18:02.987390 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.987365 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-service-ca\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:02.987443 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.987387 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-config" (OuterVolumeSpecName: "console-config") pod "15059af9-a69f-4880-b5a8-1d1a3fd12ee2" (UID: "15059af9-a69f-4880-b5a8-1d1a3fd12ee2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:18:02.987499 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.987471 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "15059af9-a69f-4880-b5a8-1d1a3fd12ee2" (UID: "15059af9-a69f-4880-b5a8-1d1a3fd12ee2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:18:02.987709 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.987691 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "15059af9-a69f-4880-b5a8-1d1a3fd12ee2" (UID: "15059af9-a69f-4880-b5a8-1d1a3fd12ee2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 04:18:02.989216 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.989194 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "15059af9-a69f-4880-b5a8-1d1a3fd12ee2" (UID: "15059af9-a69f-4880-b5a8-1d1a3fd12ee2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:18:02.989651 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.989630 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-kube-api-access-88gxd" (OuterVolumeSpecName: "kube-api-access-88gxd") pod "15059af9-a69f-4880-b5a8-1d1a3fd12ee2" (UID: "15059af9-a69f-4880-b5a8-1d1a3fd12ee2"). InnerVolumeSpecName "kube-api-access-88gxd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:18:02.989709 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:02.989650 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "15059af9-a69f-4880-b5a8-1d1a3fd12ee2" (UID: "15059af9-a69f-4880-b5a8-1d1a3fd12ee2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:18:03.087919 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.087865 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:03.087919 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.087912 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:03.087919 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.087922 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-trusted-ca-bundle\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:03.087919 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.087931 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88gxd\" (UniqueName: \"kubernetes.io/projected/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-kube-api-access-88gxd\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:03.087919 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.087940 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-oauth-serving-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:03.088228 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.087949 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15059af9-a69f-4880-b5a8-1d1a3fd12ee2-console-oauth-config\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:03.702832 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.702798 2584 generic.go:356] "Generic (PLEG): container finished" podID="15059af9-a69f-4880-b5a8-1d1a3fd12ee2" containerID="33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b" exitCode=0 Jul 09 04:18:03.703301 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.702843 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-857f44b44f-rwjr4" Jul 09 04:18:03.703301 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.702887 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f44b44f-rwjr4" event={"ID":"15059af9-a69f-4880-b5a8-1d1a3fd12ee2","Type":"ContainerDied","Data":"33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b"} Jul 09 04:18:03.703301 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.702946 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-857f44b44f-rwjr4" event={"ID":"15059af9-a69f-4880-b5a8-1d1a3fd12ee2","Type":"ContainerDied","Data":"e63e25aac08f141e46926cb55e9374f79d0c9d80efad503c02256dacaef2f65e"} Jul 09 04:18:03.703301 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.702977 2584 scope.go:117] "RemoveContainer" containerID="33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b" Jul 09 04:18:03.711758 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.711740 2584 scope.go:117] "RemoveContainer" containerID="33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b" Jul 09 04:18:03.712080 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:18:03.712061 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b\": container with ID starting with 33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b not found: ID does not exist" containerID="33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b" Jul 09 04:18:03.712130 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.712090 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b"} err="failed to get container status \"33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b\": rpc error: code = NotFound desc = could not find container \"33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b\": container with ID starting with 33e9198a38c29d77c881f1bf54df4f5d19cb9369692bced871784d79de84622b not found: ID does not exist" Jul 09 04:18:03.721751 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.721719 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-857f44b44f-rwjr4"] Jul 09 04:18:03.730380 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:03.730348 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-857f44b44f-rwjr4"] Jul 09 04:18:05.231225 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:05.231195 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15059af9-a69f-4880-b5a8-1d1a3fd12ee2" path="/var/lib/kubelet/pods/15059af9-a69f-4880-b5a8-1d1a3fd12ee2/volumes" Jul 09 04:18:38.532745 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.532708 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7nnr4"] Jul 09 04:18:38.533230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.533025 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15059af9-a69f-4880-b5a8-1d1a3fd12ee2" containerName="console" Jul 09 04:18:38.533230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.533037 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="15059af9-a69f-4880-b5a8-1d1a3fd12ee2" containerName="console" Jul 09 04:18:38.533230 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.533113 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="15059af9-a69f-4880-b5a8-1d1a3fd12ee2" containerName="console" Jul 09 04:18:38.537158 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.537135 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7nnr4" Jul 09 04:18:38.539632 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.539604 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 04:18:38.539764 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.539666 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7nnr4"] Jul 09 04:18:38.540628 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.540611 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4xstw\"" Jul 09 04:18:38.540736 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.540663 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 04:18:38.703842 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.703802 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/6773f7c6-9261-4821-9ce3-dbec75bb453b-kube-api-access-52wgk\") pod \"authorino-58cd955675-7nnr4\" (UID: \"6773f7c6-9261-4821-9ce3-dbec75bb453b\") " pod="kuadrant-system/authorino-58cd955675-7nnr4" Jul 09 04:18:38.738405 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.738366 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-v6956"] Jul 09 04:18:38.741747 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.741721 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v6956" Jul 09 04:18:38.748422 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.748390 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v6956"] Jul 09 04:18:38.804939 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.804835 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/6773f7c6-9261-4821-9ce3-dbec75bb453b-kube-api-access-52wgk\") pod \"authorino-58cd955675-7nnr4\" (UID: \"6773f7c6-9261-4821-9ce3-dbec75bb453b\") " pod="kuadrant-system/authorino-58cd955675-7nnr4" Jul 09 04:18:38.813248 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.813213 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/6773f7c6-9261-4821-9ce3-dbec75bb453b-kube-api-access-52wgk\") pod \"authorino-58cd955675-7nnr4\" (UID: \"6773f7c6-9261-4821-9ce3-dbec75bb453b\") " pod="kuadrant-system/authorino-58cd955675-7nnr4" Jul 09 04:18:38.847797 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.847761 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7nnr4" Jul 09 04:18:38.906073 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.906044 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlbs8\" (UniqueName: \"kubernetes.io/projected/478c41ac-6d24-4c8a-b501-531423e5a28f-kube-api-access-nlbs8\") pod \"authorino-75d556bd77-v6956\" (UID: \"478c41ac-6d24-4c8a-b501-531423e5a28f\") " pod="kuadrant-system/authorino-75d556bd77-v6956" Jul 09 04:18:38.981231 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.981209 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7nnr4"] Jul 09 04:18:38.983849 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:18:38.983813 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6773f7c6_9261_4821_9ce3_dbec75bb453b.slice/crio-86375eff52307418f909d7a0703827afe6e0e4ade37f9ce5630994cd8e1058d4 WatchSource:0}: Error finding container 86375eff52307418f909d7a0703827afe6e0e4ade37f9ce5630994cd8e1058d4: Status 404 returned error can't find the container with id 86375eff52307418f909d7a0703827afe6e0e4ade37f9ce5630994cd8e1058d4 Jul 09 04:18:38.985154 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:38.985136 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 04:18:39.007285 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:39.007255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlbs8\" (UniqueName: \"kubernetes.io/projected/478c41ac-6d24-4c8a-b501-531423e5a28f-kube-api-access-nlbs8\") pod \"authorino-75d556bd77-v6956\" (UID: \"478c41ac-6d24-4c8a-b501-531423e5a28f\") " pod="kuadrant-system/authorino-75d556bd77-v6956" Jul 09 04:18:39.014775 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:39.014750 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlbs8\" (UniqueName: \"kubernetes.io/projected/478c41ac-6d24-4c8a-b501-531423e5a28f-kube-api-access-nlbs8\") pod \"authorino-75d556bd77-v6956\" (UID: \"478c41ac-6d24-4c8a-b501-531423e5a28f\") " pod="kuadrant-system/authorino-75d556bd77-v6956" Jul 09 04:18:39.051701 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:39.051668 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v6956" Jul 09 04:18:39.180538 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:39.180491 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v6956"] Jul 09 04:18:39.182900 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:18:39.182878 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod478c41ac_6d24_4c8a_b501_531423e5a28f.slice/crio-28686f7860b38c91a659dd7a906b9c89d97bee8ed07961c1e4dcf82faa05867c WatchSource:0}: Error finding container 28686f7860b38c91a659dd7a906b9c89d97bee8ed07961c1e4dcf82faa05867c: Status 404 returned error can't find the container with id 28686f7860b38c91a659dd7a906b9c89d97bee8ed07961c1e4dcf82faa05867c Jul 09 04:18:39.872097 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:39.872054 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7nnr4" event={"ID":"6773f7c6-9261-4821-9ce3-dbec75bb453b","Type":"ContainerStarted","Data":"86375eff52307418f909d7a0703827afe6e0e4ade37f9ce5630994cd8e1058d4"} Jul 09 04:18:39.874887 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:39.874806 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v6956" event={"ID":"478c41ac-6d24-4c8a-b501-531423e5a28f","Type":"ContainerStarted","Data":"28686f7860b38c91a659dd7a906b9c89d97bee8ed07961c1e4dcf82faa05867c"} Jul 09 04:18:42.895717 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:42.895681 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v6956" event={"ID":"478c41ac-6d24-4c8a-b501-531423e5a28f","Type":"ContainerStarted","Data":"b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9"} Jul 09 04:18:42.897078 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:42.897054 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7nnr4" event={"ID":"6773f7c6-9261-4821-9ce3-dbec75bb453b","Type":"ContainerStarted","Data":"d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67"} Jul 09 04:18:42.910219 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:42.910172 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-v6956" podStartSLOduration=1.56527846 podStartE2EDuration="4.910158411s" totalImagesPullingTime="3.344879951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:18:42.909664973 +0000 UTC m=+680.146635152" watchObservedRunningTime="2026-07-09 04:18:42.910158411 +0000 UTC m=+680.147128609" Jul 09 04:18:42.923108 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:42.923061 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7nnr4" podStartSLOduration=1.368386378 podStartE2EDuration="4.923045375s" totalImagesPullingTime="3.554658997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:18:42.922018891 +0000 UTC m=+680.158989069" watchObservedRunningTime="2026-07-09 04:18:42.923045375 +0000 UTC m=+680.160015553" Jul 09 04:18:42.942967 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:42.942934 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7nnr4"] Jul 09 04:18:44.905324 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:44.905268 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7nnr4" podUID="6773f7c6-9261-4821-9ce3-dbec75bb453b" containerName="authorino" containerID="cri-o://d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67" gracePeriod=30 Jul 09 04:18:45.148171 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.148141 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7nnr4" Jul 09 04:18:45.261332 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.261248 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/6773f7c6-9261-4821-9ce3-dbec75bb453b-kube-api-access-52wgk\") pod \"6773f7c6-9261-4821-9ce3-dbec75bb453b\" (UID: \"6773f7c6-9261-4821-9ce3-dbec75bb453b\") " Jul 09 04:18:45.263498 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.263471 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6773f7c6-9261-4821-9ce3-dbec75bb453b-kube-api-access-52wgk" (OuterVolumeSpecName: "kube-api-access-52wgk") pod "6773f7c6-9261-4821-9ce3-dbec75bb453b" (UID: "6773f7c6-9261-4821-9ce3-dbec75bb453b"). InnerVolumeSpecName "kube-api-access-52wgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:18:45.362714 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.362676 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52wgk\" (UniqueName: \"kubernetes.io/projected/6773f7c6-9261-4821-9ce3-dbec75bb453b-kube-api-access-52wgk\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:18:45.910568 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.910535 2584 generic.go:356] "Generic (PLEG): container finished" podID="6773f7c6-9261-4821-9ce3-dbec75bb453b" containerID="d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67" exitCode=0 Jul 09 04:18:45.911023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.910585 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7nnr4" Jul 09 04:18:45.911023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.910626 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7nnr4" event={"ID":"6773f7c6-9261-4821-9ce3-dbec75bb453b","Type":"ContainerDied","Data":"d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67"} Jul 09 04:18:45.911023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.910677 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7nnr4" event={"ID":"6773f7c6-9261-4821-9ce3-dbec75bb453b","Type":"ContainerDied","Data":"86375eff52307418f909d7a0703827afe6e0e4ade37f9ce5630994cd8e1058d4"} Jul 09 04:18:45.911023 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.910703 2584 scope.go:117] "RemoveContainer" containerID="d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67" Jul 09 04:18:45.919173 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.919156 2584 scope.go:117] "RemoveContainer" containerID="d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67" Jul 09 04:18:45.919459 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:18:45.919437 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67\": container with ID starting with d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67 not found: ID does not exist" containerID="d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67" Jul 09 04:18:45.919503 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.919469 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67"} err="failed to get container status \"d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67\": rpc error: code = NotFound desc = could not find container \"d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67\": container with ID starting with d1edb3adb24146026a43bced62484749bb02aa4805cd6cc9061f6ef26547dc67 not found: ID does not exist" Jul 09 04:18:45.930039 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.930014 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7nnr4"] Jul 09 04:18:45.933358 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:45.933335 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7nnr4"] Jul 09 04:18:47.231755 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:18:47.231718 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6773f7c6-9261-4821-9ce3-dbec75bb453b" path="/var/lib/kubelet/pods/6773f7c6-9261-4821-9ce3-dbec75bb453b/volumes" Jul 09 04:19:10.864668 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:10.864622 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tjp4g"] Jul 09 04:19:10.865121 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:10.865065 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6773f7c6-9261-4821-9ce3-dbec75bb453b" containerName="authorino" Jul 09 04:19:10.865121 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:10.865080 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6773f7c6-9261-4821-9ce3-dbec75bb453b" containerName="authorino" Jul 09 04:19:10.865190 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:10.865174 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6773f7c6-9261-4821-9ce3-dbec75bb453b" containerName="authorino" Jul 09 04:19:10.869697 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:10.869677 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tjp4g" Jul 09 04:19:10.873488 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:10.873463 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tjp4g"] Jul 09 04:19:10.973970 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:10.973936 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnsg5\" (UniqueName: \"kubernetes.io/projected/b904061c-8a64-4382-88f3-804821048804-kube-api-access-vnsg5\") pod \"authorino-5d4b67c446-tjp4g\" (UID: \"b904061c-8a64-4382-88f3-804821048804\") " pod="kuadrant-system/authorino-5d4b67c446-tjp4g" Jul 09 04:19:11.061024 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.060989 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tjp4g"] Jul 09 04:19:11.061263 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:19:11.061245 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vnsg5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-tjp4g" podUID="b904061c-8a64-4382-88f3-804821048804" Jul 09 04:19:11.075185 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.075151 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnsg5\" (UniqueName: \"kubernetes.io/projected/b904061c-8a64-4382-88f3-804821048804-kube-api-access-vnsg5\") pod \"authorino-5d4b67c446-tjp4g\" (UID: \"b904061c-8a64-4382-88f3-804821048804\") " pod="kuadrant-system/authorino-5d4b67c446-tjp4g" Jul 09 04:19:11.083627 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.083594 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnsg5\" (UniqueName: \"kubernetes.io/projected/b904061c-8a64-4382-88f3-804821048804-kube-api-access-vnsg5\") pod \"authorino-5d4b67c446-tjp4g\" (UID: \"b904061c-8a64-4382-88f3-804821048804\") " pod="kuadrant-system/authorino-5d4b67c446-tjp4g" Jul 09 04:19:11.087000 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.086980 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-j5nd4"] Jul 09 04:19:11.090207 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.090190 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:11.092601 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.092584 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 04:19:11.099685 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.099662 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-j5nd4"] Jul 09 04:19:11.120052 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.119978 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-j5nd4"] Jul 09 04:19:11.120238 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:19:11.120219 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-kxw4g tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-kxw4g tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-j5nd4" podUID="5a8144a6-0060-4993-a3a6-9d90a3c8f2c8" Jul 09 04:19:11.155027 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.154999 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7545f9f9cc-44pvf"] Jul 09 04:19:11.158146 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.158130 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.165247 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.165218 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7545f9f9cc-44pvf"] Jul 09 04:19:11.176042 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.176015 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-tls-cert\") pod \"authorino-76877994-j5nd4\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:11.176167 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.176060 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxw4g\" (UniqueName: \"kubernetes.io/projected/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-kube-api-access-kxw4g\") pod \"authorino-76877994-j5nd4\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:11.276720 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.276688 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/853f520a-b6d0-4cde-8c82-49ed039cd8b5-tls-cert\") pod \"authorino-7545f9f9cc-44pvf\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.276937 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.276743 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxw4g\" (UniqueName: \"kubernetes.io/projected/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-kube-api-access-kxw4g\") pod \"authorino-76877994-j5nd4\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:11.276937 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.276805 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn94l\" (UniqueName: \"kubernetes.io/projected/853f520a-b6d0-4cde-8c82-49ed039cd8b5-kube-api-access-gn94l\") pod \"authorino-7545f9f9cc-44pvf\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.276937 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.276846 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-tls-cert\") pod \"authorino-76877994-j5nd4\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:11.279347 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.279327 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-tls-cert\") pod \"authorino-76877994-j5nd4\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:11.284244 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.284219 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxw4g\" (UniqueName: \"kubernetes.io/projected/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-kube-api-access-kxw4g\") pod \"authorino-76877994-j5nd4\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:11.377892 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.377806 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/853f520a-b6d0-4cde-8c82-49ed039cd8b5-tls-cert\") pod \"authorino-7545f9f9cc-44pvf\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.378033 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.377917 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn94l\" (UniqueName: \"kubernetes.io/projected/853f520a-b6d0-4cde-8c82-49ed039cd8b5-kube-api-access-gn94l\") pod \"authorino-7545f9f9cc-44pvf\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.380411 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.380390 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/853f520a-b6d0-4cde-8c82-49ed039cd8b5-tls-cert\") pod \"authorino-7545f9f9cc-44pvf\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.385436 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.385403 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn94l\" (UniqueName: \"kubernetes.io/projected/853f520a-b6d0-4cde-8c82-49ed039cd8b5-kube-api-access-gn94l\") pod \"authorino-7545f9f9cc-44pvf\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.467854 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.467799 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:19:11.599103 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:11.599055 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7545f9f9cc-44pvf"] Jul 09 04:19:11.601783 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:19:11.601754 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod853f520a_b6d0_4cde_8c82_49ed039cd8b5.slice/crio-f7916acd9c23d040f67e02a1aa2e52d8a06642199af3effa7f2042d11a391cfb WatchSource:0}: Error finding container f7916acd9c23d040f67e02a1aa2e52d8a06642199af3effa7f2042d11a391cfb: Status 404 returned error can't find the container with id f7916acd9c23d040f67e02a1aa2e52d8a06642199af3effa7f2042d11a391cfb Jul 09 04:19:12.021022 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.020988 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" event={"ID":"853f520a-b6d0-4cde-8c82-49ed039cd8b5","Type":"ContainerStarted","Data":"f7916acd9c23d040f67e02a1aa2e52d8a06642199af3effa7f2042d11a391cfb"} Jul 09 04:19:12.021022 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.021006 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tjp4g" Jul 09 04:19:12.021022 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.021000 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:12.029365 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.029333 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tjp4g" Jul 09 04:19:12.031622 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.031600 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:12.085591 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.085570 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnsg5\" (UniqueName: \"kubernetes.io/projected/b904061c-8a64-4382-88f3-804821048804-kube-api-access-vnsg5\") pod \"b904061c-8a64-4382-88f3-804821048804\" (UID: \"b904061c-8a64-4382-88f3-804821048804\") " Jul 09 04:19:12.085702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.085598 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxw4g\" (UniqueName: \"kubernetes.io/projected/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-kube-api-access-kxw4g\") pod \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " Jul 09 04:19:12.085702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.085632 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-tls-cert\") pod \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\" (UID: \"5a8144a6-0060-4993-a3a6-9d90a3c8f2c8\") " Jul 09 04:19:12.087483 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.087449 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b904061c-8a64-4382-88f3-804821048804-kube-api-access-vnsg5" (OuterVolumeSpecName: "kube-api-access-vnsg5") pod "b904061c-8a64-4382-88f3-804821048804" (UID: "b904061c-8a64-4382-88f3-804821048804"). InnerVolumeSpecName "kube-api-access-vnsg5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:19:12.087690 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.087626 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-kube-api-access-kxw4g" (OuterVolumeSpecName: "kube-api-access-kxw4g") pod "5a8144a6-0060-4993-a3a6-9d90a3c8f2c8" (UID: "5a8144a6-0060-4993-a3a6-9d90a3c8f2c8"). InnerVolumeSpecName "kube-api-access-kxw4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:19:12.087690 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.087650 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5a8144a6-0060-4993-a3a6-9d90a3c8f2c8" (UID: "5a8144a6-0060-4993-a3a6-9d90a3c8f2c8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:19:12.186744 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.186711 2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-tls-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:19:12.186744 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.186740 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnsg5\" (UniqueName: \"kubernetes.io/projected/b904061c-8a64-4382-88f3-804821048804-kube-api-access-vnsg5\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:19:12.186744 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:12.186750 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxw4g\" (UniqueName: \"kubernetes.io/projected/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8-kube-api-access-kxw4g\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:19:13.028113 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.028069 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" event={"ID":"853f520a-b6d0-4cde-8c82-49ed039cd8b5","Type":"ContainerStarted","Data":"e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13"} Jul 09 04:19:13.028113 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.028096 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-j5nd4" Jul 09 04:19:13.028113 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.028106 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tjp4g" Jul 09 04:19:13.049579 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.049504 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" podStartSLOduration=1.576453773 podStartE2EDuration="2.049486699s" totalImagesPullingTime="473.032926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:19:13.048080933 +0000 UTC m=+710.285051121" watchObservedRunningTime="2026-07-09 04:19:13.049486699 +0000 UTC m=+710.286456876" Jul 09 04:19:13.074662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.074621 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tjp4g"] Jul 09 04:19:13.079716 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.079687 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v6956"] Jul 09 04:19:13.079919 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.079897 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-v6956" podUID="478c41ac-6d24-4c8a-b501-531423e5a28f" containerName="authorino" containerID="cri-o://b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9" gracePeriod=30 Jul 09 04:19:13.082267 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.082246 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tjp4g"] Jul 09 04:19:13.107790 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.107748 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-j5nd4"] Jul 09 04:19:13.113836 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.113798 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-j5nd4"] Jul 09 04:19:13.231303 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.231267 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a8144a6-0060-4993-a3a6-9d90a3c8f2c8" path="/var/lib/kubelet/pods/5a8144a6-0060-4993-a3a6-9d90a3c8f2c8/volumes" Jul 09 04:19:13.231505 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.231494 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b904061c-8a64-4382-88f3-804821048804" path="/var/lib/kubelet/pods/b904061c-8a64-4382-88f3-804821048804/volumes" Jul 09 04:19:13.319124 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.319099 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v6956" Jul 09 04:19:13.397143 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.397109 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlbs8\" (UniqueName: \"kubernetes.io/projected/478c41ac-6d24-4c8a-b501-531423e5a28f-kube-api-access-nlbs8\") pod \"478c41ac-6d24-4c8a-b501-531423e5a28f\" (UID: \"478c41ac-6d24-4c8a-b501-531423e5a28f\") " Jul 09 04:19:13.399336 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.399315 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/478c41ac-6d24-4c8a-b501-531423e5a28f-kube-api-access-nlbs8" (OuterVolumeSpecName: "kube-api-access-nlbs8") pod "478c41ac-6d24-4c8a-b501-531423e5a28f" (UID: "478c41ac-6d24-4c8a-b501-531423e5a28f"). InnerVolumeSpecName "kube-api-access-nlbs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:19:13.498232 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:13.498194 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlbs8\" (UniqueName: \"kubernetes.io/projected/478c41ac-6d24-4c8a-b501-531423e5a28f-kube-api-access-nlbs8\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:19:14.034420 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.034387 2584 generic.go:356] "Generic (PLEG): container finished" podID="478c41ac-6d24-4c8a-b501-531423e5a28f" containerID="b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9" exitCode=0 Jul 09 04:19:14.034904 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.034440 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-v6956" Jul 09 04:19:14.034904 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.034476 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v6956" event={"ID":"478c41ac-6d24-4c8a-b501-531423e5a28f","Type":"ContainerDied","Data":"b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9"} Jul 09 04:19:14.034904 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.034543 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-v6956" event={"ID":"478c41ac-6d24-4c8a-b501-531423e5a28f","Type":"ContainerDied","Data":"28686f7860b38c91a659dd7a906b9c89d97bee8ed07961c1e4dcf82faa05867c"} Jul 09 04:19:14.034904 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.034554 2584 scope.go:117] "RemoveContainer" containerID="b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9" Jul 09 04:19:14.042613 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.042595 2584 scope.go:117] "RemoveContainer" containerID="b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9" Jul 09 04:19:14.042883 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:19:14.042865 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9\": container with ID starting with b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9 not found: ID does not exist" containerID="b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9" Jul 09 04:19:14.042931 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.042907 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9"} err="failed to get container status \"b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9\": rpc error: code = NotFound desc = could not find container \"b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9\": container with ID starting with b9751864a7dbbdbcdb3479a27a373b201d3098aa9938dea204b4132cbc3968e9 not found: ID does not exist" Jul 09 04:19:14.054273 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.054248 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v6956"] Jul 09 04:19:14.058172 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:14.058150 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-v6956"] Jul 09 04:19:15.230419 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:19:15.230387 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="478c41ac-6d24-4c8a-b501-531423e5a28f" path="/var/lib/kubelet/pods/478c41ac-6d24-4c8a-b501-531423e5a28f/volumes" Jul 09 04:20:35.559075 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.559045 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8"] Jul 09 04:20:35.559577 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.559366 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="478c41ac-6d24-4c8a-b501-531423e5a28f" containerName="authorino" Jul 09 04:20:35.559577 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.559378 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="478c41ac-6d24-4c8a-b501-531423e5a28f" containerName="authorino" Jul 09 04:20:35.559577 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.559446 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="478c41ac-6d24-4c8a-b501-531423e5a28f" containerName="authorino" Jul 09 04:20:35.561261 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.561244 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.564945 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.564921 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 04:20:35.564945 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.564940 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hnnjh\"" Jul 09 04:20:35.565135 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.564950 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 04:20:35.565135 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.564929 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 09 04:20:35.571954 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.571933 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8"] Jul 09 04:20:35.618247 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.618211 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98151876-c4e4-4196-b1e2-49209b0a63ee-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.618247 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.618253 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.618445 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.618277 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.618445 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.618344 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.618445 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.618372 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-429cp\" (UniqueName: \"kubernetes.io/projected/98151876-c4e4-4196-b1e2-49209b0a63ee-kube-api-access-429cp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.618445 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.618404 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719192 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719156 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719192 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719415 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719220 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719415 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719254 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-429cp\" (UniqueName: \"kubernetes.io/projected/98151876-c4e4-4196-b1e2-49209b0a63ee-kube-api-access-429cp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719415 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719306 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719415 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719367 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98151876-c4e4-4196-b1e2-49209b0a63ee-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719718 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719684 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719784 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719761 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.719828 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.719808 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.721751 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.721734 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98151876-c4e4-4196-b1e2-49209b0a63ee-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.721911 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.721895 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98151876-c4e4-4196-b1e2-49209b0a63ee-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.727031 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.727001 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-429cp\" (UniqueName: \"kubernetes.io/projected/98151876-c4e4-4196-b1e2-49209b0a63ee-kube-api-access-429cp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2zfh8\" (UID: \"98151876-c4e4-4196-b1e2-49209b0a63ee\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:35.872708 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:35.872596 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:36.006102 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:36.006069 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8"] Jul 09 04:20:36.010217 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:20:36.010189 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98151876_c4e4_4196_b1e2_49209b0a63ee.slice/crio-565e59d38bb5a1a348e95939790162b3a99c3fc9882e9328cbb649cb8eb57607 WatchSource:0}: Error finding container 565e59d38bb5a1a348e95939790162b3a99c3fc9882e9328cbb649cb8eb57607: Status 404 returned error can't find the container with id 565e59d38bb5a1a348e95939790162b3a99c3fc9882e9328cbb649cb8eb57607 Jul 09 04:20:36.393073 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:36.393032 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" event={"ID":"98151876-c4e4-4196-b1e2-49209b0a63ee","Type":"ContainerStarted","Data":"565e59d38bb5a1a348e95939790162b3a99c3fc9882e9328cbb649cb8eb57607"} Jul 09 04:20:37.400286 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:37.400240 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" event={"ID":"98151876-c4e4-4196-b1e2-49209b0a63ee","Type":"ContainerStarted","Data":"11bceb079d721a205fb61793a4091c961373a2d7a1f526372d395b00333cdd11"} Jul 09 04:20:37.418582 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:37.418507 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" podStartSLOduration=1.4476313699999999 podStartE2EDuration="2.418489321s" totalImagesPullingTime="970.857951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:20:37.417630115 +0000 UTC m=+794.654600294" watchObservedRunningTime="2026-07-09 04:20:37.418489321 +0000 UTC m=+794.655459500" Jul 09 04:20:45.873158 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:45.873068 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:45.873158 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:45.873147 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:45.885664 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:45.885638 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:20:46.449955 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:20:46.449925 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2zfh8" Jul 09 04:21:17.670551 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.670485 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl"] Jul 09 04:21:17.673928 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.673904 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.676669 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.676650 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 04:21:17.684063 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.684035 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl"] Jul 09 04:21:17.693743 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.693721 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.693866 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.693751 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.693866 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.693789 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be9d319c-92d9-4bea-966a-d8871c8f1f68-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.693980 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.693906 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.693980 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.693944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.694065 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.694039 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crt4n\" (UniqueName: \"kubernetes.io/projected/be9d319c-92d9-4bea-966a-d8871c8f1f68-kube-api-access-crt4n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.794814 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.794776 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be9d319c-92d9-4bea-966a-d8871c8f1f68-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.794814 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.794820 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.795052 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.794838 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.795052 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.794867 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crt4n\" (UniqueName: \"kubernetes.io/projected/be9d319c-92d9-4bea-966a-d8871c8f1f68-kube-api-access-crt4n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.795052 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.794938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.795052 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.794965 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.795250 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.795194 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.795310 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.795265 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.795369 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.795319 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.797281 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.797251 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be9d319c-92d9-4bea-966a-d8871c8f1f68-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.797463 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.797447 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be9d319c-92d9-4bea-966a-d8871c8f1f68-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.802620 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.802594 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crt4n\" (UniqueName: \"kubernetes.io/projected/be9d319c-92d9-4bea-966a-d8871c8f1f68-kube-api-access-crt4n\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl\" (UID: \"be9d319c-92d9-4bea-966a-d8871c8f1f68\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:17.983580 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:17.983474 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:18.117313 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:18.117287 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl"] Jul 09 04:21:18.119720 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:21:18.119689 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe9d319c_92d9_4bea_966a_d8871c8f1f68.slice/crio-29ad25a72a8e0c3e777e5c2cda2fe63fe0b5645c5d526b8be1943ecc1a3434ce WatchSource:0}: Error finding container 29ad25a72a8e0c3e777e5c2cda2fe63fe0b5645c5d526b8be1943ecc1a3434ce: Status 404 returned error can't find the container with id 29ad25a72a8e0c3e777e5c2cda2fe63fe0b5645c5d526b8be1943ecc1a3434ce Jul 09 04:21:18.584437 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:18.584348 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" event={"ID":"be9d319c-92d9-4bea-966a-d8871c8f1f68","Type":"ContainerStarted","Data":"94fc242db41d56c87220de9be46bb9f4531281a9099e7c2184b313a3a4bd3a77"} Jul 09 04:21:18.584437 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:18.584390 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" event={"ID":"be9d319c-92d9-4bea-966a-d8871c8f1f68","Type":"ContainerStarted","Data":"29ad25a72a8e0c3e777e5c2cda2fe63fe0b5645c5d526b8be1943ecc1a3434ce"} Jul 09 04:21:18.601090 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:18.601037 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" podStartSLOduration=1.429056195 podStartE2EDuration="1.601019707s" totalImagesPullingTime="171.963512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:21:18.599301256 +0000 UTC m=+835.836271433" watchObservedRunningTime="2026-07-09 04:21:18.601019707 +0000 UTC m=+835.837989885" Jul 09 04:21:27.984130 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:27.984083 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:27.984130 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:27.984144 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:27.996807 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:27.996773 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:21:28.639874 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:21:28.639841 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl" Jul 09 04:22:12.320564 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.320511 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-68cc768756-dr5cr"] Jul 09 04:22:12.324010 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.323991 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.332412 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.332384 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68cc768756-dr5cr"] Jul 09 04:22:12.354071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.353820 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdhr6\" (UniqueName: \"kubernetes.io/projected/744118a4-270a-4852-a22b-e39d584b025d-kube-api-access-rdhr6\") pod \"authorino-68cc768756-dr5cr\" (UID: \"744118a4-270a-4852-a22b-e39d584b025d\") " pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.354071 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.354045 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/744118a4-270a-4852-a22b-e39d584b025d-tls-cert\") pod \"authorino-68cc768756-dr5cr\" (UID: \"744118a4-270a-4852-a22b-e39d584b025d\") " pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.454841 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.454808 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/744118a4-270a-4852-a22b-e39d584b025d-tls-cert\") pod \"authorino-68cc768756-dr5cr\" (UID: \"744118a4-270a-4852-a22b-e39d584b025d\") " pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.455007 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.454878 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdhr6\" (UniqueName: \"kubernetes.io/projected/744118a4-270a-4852-a22b-e39d584b025d-kube-api-access-rdhr6\") pod \"authorino-68cc768756-dr5cr\" (UID: \"744118a4-270a-4852-a22b-e39d584b025d\") " pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.457421 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.457398 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/744118a4-270a-4852-a22b-e39d584b025d-tls-cert\") pod \"authorino-68cc768756-dr5cr\" (UID: \"744118a4-270a-4852-a22b-e39d584b025d\") " pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.462288 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.462259 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdhr6\" (UniqueName: \"kubernetes.io/projected/744118a4-270a-4852-a22b-e39d584b025d-kube-api-access-rdhr6\") pod \"authorino-68cc768756-dr5cr\" (UID: \"744118a4-270a-4852-a22b-e39d584b025d\") " pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.656531 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.656496 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-68cc768756-dr5cr" Jul 09 04:22:12.795162 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.794991 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-68cc768756-dr5cr"] Jul 09 04:22:12.797748 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:22:12.797719 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod744118a4_270a_4852_a22b_e39d584b025d.slice/crio-5ae527edfb12f85e437b0922d5f97216fa9582cd730e542533cc065828180e63 WatchSource:0}: Error finding container 5ae527edfb12f85e437b0922d5f97216fa9582cd730e542533cc065828180e63: Status 404 returned error can't find the container with id 5ae527edfb12f85e437b0922d5f97216fa9582cd730e542533cc065828180e63 Jul 09 04:22:12.838232 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:12.838196 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68cc768756-dr5cr" event={"ID":"744118a4-270a-4852-a22b-e39d584b025d","Type":"ContainerStarted","Data":"5ae527edfb12f85e437b0922d5f97216fa9582cd730e542533cc065828180e63"} Jul 09 04:22:13.846930 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:13.846886 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-68cc768756-dr5cr" event={"ID":"744118a4-270a-4852-a22b-e39d584b025d","Type":"ContainerStarted","Data":"7f10d3e0101faa98674477c0e28bbbf75c314b97aa1a26e89efeb307fdec60ac"} Jul 09 04:22:13.934694 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:13.934639 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-68cc768756-dr5cr" podStartSLOduration=1.522871519 podStartE2EDuration="1.934620926s" totalImagesPullingTime="411.749407ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 04:22:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:22:13.921160132 +0000 UTC m=+891.158130328" watchObservedRunningTime="2026-07-09 04:22:13.934620926 +0000 UTC m=+891.171591157" Jul 09 04:22:13.993451 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:13.993407 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7545f9f9cc-44pvf"] Jul 09 04:22:13.993719 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:13.993690 2584 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" podUID="853f520a-b6d0-4cde-8c82-49ed039cd8b5" containerName="authorino" containerID="cri-o://e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13" gracePeriod=30 Jul 09 04:22:14.252331 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.252303 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:22:14.276886 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.276855 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn94l\" (UniqueName: \"kubernetes.io/projected/853f520a-b6d0-4cde-8c82-49ed039cd8b5-kube-api-access-gn94l\") pod \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " Jul 09 04:22:14.277046 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.276926 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/853f520a-b6d0-4cde-8c82-49ed039cd8b5-tls-cert\") pod \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\" (UID: \"853f520a-b6d0-4cde-8c82-49ed039cd8b5\") " Jul 09 04:22:14.279333 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.279299 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853f520a-b6d0-4cde-8c82-49ed039cd8b5-kube-api-access-gn94l" (OuterVolumeSpecName: "kube-api-access-gn94l") pod "853f520a-b6d0-4cde-8c82-49ed039cd8b5" (UID: "853f520a-b6d0-4cde-8c82-49ed039cd8b5"). InnerVolumeSpecName "kube-api-access-gn94l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 04:22:14.287385 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.287347 2584 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/853f520a-b6d0-4cde-8c82-49ed039cd8b5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "853f520a-b6d0-4cde-8c82-49ed039cd8b5" (UID: "853f520a-b6d0-4cde-8c82-49ed039cd8b5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 04:22:14.377954 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.377874 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn94l\" (UniqueName: \"kubernetes.io/projected/853f520a-b6d0-4cde-8c82-49ed039cd8b5-kube-api-access-gn94l\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:22:14.377954 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.377902 2584 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/853f520a-b6d0-4cde-8c82-49ed039cd8b5-tls-cert\") on node \"ip-10-0-130-250.ec2.internal\" DevicePath \"\"" Jul 09 04:22:14.853197 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.853162 2584 generic.go:356] "Generic (PLEG): container finished" podID="853f520a-b6d0-4cde-8c82-49ed039cd8b5" containerID="e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13" exitCode=0 Jul 09 04:22:14.853647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.853211 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" Jul 09 04:22:14.853647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.853259 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" event={"ID":"853f520a-b6d0-4cde-8c82-49ed039cd8b5","Type":"ContainerDied","Data":"e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13"} Jul 09 04:22:14.853647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.853317 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7545f9f9cc-44pvf" event={"ID":"853f520a-b6d0-4cde-8c82-49ed039cd8b5","Type":"ContainerDied","Data":"f7916acd9c23d040f67e02a1aa2e52d8a06642199af3effa7f2042d11a391cfb"} Jul 09 04:22:14.853647 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.853345 2584 scope.go:117] "RemoveContainer" containerID="e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13" Jul 09 04:22:14.862783 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.862766 2584 scope.go:117] "RemoveContainer" containerID="e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13" Jul 09 04:22:14.863110 ip-10-0-130-250 kubenswrapper[2584]: E0709 04:22:14.863092 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13\": container with ID starting with e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13 not found: ID does not exist" containerID="e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13" Jul 09 04:22:14.863163 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.863118 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13"} err="failed to get container status \"e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13\": rpc error: code = NotFound desc = could not find container \"e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13\": container with ID starting with e9b4677a4a620e90238c55349d8acc1ae7a02699e67b70e31472221250680f13 not found: ID does not exist" Jul 09 04:22:14.873681 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.873655 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7545f9f9cc-44pvf"] Jul 09 04:22:14.879718 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:14.879695 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7545f9f9cc-44pvf"] Jul 09 04:22:15.230417 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:22:15.230388 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="853f520a-b6d0-4cde-8c82-49ed039cd8b5" path="/var/lib/kubelet/pods/853f520a-b6d0-4cde-8c82-49ed039cd8b5/volumes" Jul 09 04:54:12.646477 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:12.646443 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68cc768756-dr5cr_744118a4-270a-4852-a22b-e39d584b025d/authorino/0.log" Jul 09 04:54:17.628891 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:17.628856 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fhmnf_a3545f05-462f-4a55-8fba-01b632f438a2/server/0.log" Jul 09 04:54:17.745353 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:17.745325 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j8cdg_d74231e1-0159-4ab1-8d87-6dee366ea477/manager/0.log" Jul 09 04:54:18.094391 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:18.094363 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-wfxxn_d02ffa15-e098-4367-92fd-0e0c9167df22/copy-manifests/0.log" Jul 09 04:54:18.103537 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:18.103490 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-wfxxn_d02ffa15-e098-4367-92fd-0e0c9167df22/manager/0.log" Jul 09 04:54:20.056334 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:20.056286 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68cc768756-dr5cr_744118a4-270a-4852-a22b-e39d584b025d/authorino/0.log" Jul 09 04:54:21.191849 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:21.191820 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48c4dm_e357a8ee-eb6e-45bb-80a6-c0aa15966536/istio-proxy/0.log" Jul 09 04:54:21.531556 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:21.531502 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ffb9d954-z4j2p_c025e8a3-7bea-428c-b592-85a126570924/kube-auth-proxy/0.log" Jul 09 04:54:22.547757 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:22.547723 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2zfh8_98151876-c4e4-4196-b1e2-49209b0a63ee/main/0.log" Jul 09 04:54:22.869839 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:22.869757 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-2x7hl_be9d319c-92d9-4bea-966a-d8871c8f1f68/main/0.log" Jul 09 04:54:29.871660 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:29.871634 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4dwrv_10a1edc9-d06f-4cf7-b217-9334547578b8/global-pull-secret-syncer/0.log" Jul 09 04:54:30.079981 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:30.079943 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xpm7j_2d23d0e3-49b5-4448-b5cb-25c430ab0fa5/konnectivity-agent/0.log" Jul 09 04:54:30.124041 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:30.123955 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-250.ec2.internal_f6014e45b6f02267c2f67b825f59d370/haproxy/0.log" Jul 09 04:54:35.762831 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:35.762795 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-68cc768756-dr5cr_744118a4-270a-4852-a22b-e39d584b025d/authorino/0.log" Jul 09 04:54:37.725225 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.725193 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_60667dfa-bb62-44e4-b457-fd5f194b5958/alertmanager/0.log" Jul 09 04:54:37.756823 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.756757 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_60667dfa-bb62-44e4-b457-fd5f194b5958/config-reloader/0.log" Jul 09 04:54:37.784256 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.784225 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_60667dfa-bb62-44e4-b457-fd5f194b5958/kube-rbac-proxy-web/0.log" Jul 09 04:54:37.807913 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.807885 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_60667dfa-bb62-44e4-b457-fd5f194b5958/kube-rbac-proxy/0.log" Jul 09 04:54:37.829416 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.829385 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_60667dfa-bb62-44e4-b457-fd5f194b5958/kube-rbac-proxy-metric/0.log" Jul 09 04:54:37.852278 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.852247 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_60667dfa-bb62-44e4-b457-fd5f194b5958/prom-label-proxy/0.log" Jul 09 04:54:37.879717 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.879658 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_60667dfa-bb62-44e4-b457-fd5f194b5958/init-config-reloader/0.log" Jul 09 04:54:37.929349 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:37.929317 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-94dzn_60c1f4a4-fa64-4ce8-ae82-1bcdb46bf4e3/cluster-monitoring-operator/0.log" Jul 09 04:54:38.091953 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:38.091864 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6spx_7da38a96-1760-40b4-a54c-fd5356ecf2c7/node-exporter/0.log" Jul 09 04:54:38.112375 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:38.112348 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6spx_7da38a96-1760-40b4-a54c-fd5356ecf2c7/kube-rbac-proxy/0.log" Jul 09 04:54:38.143492 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:38.143464 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-h6spx_7da38a96-1760-40b4-a54c-fd5356ecf2c7/init-textfile/0.log" Jul 09 04:54:38.602253 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:38.602223 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-6ssvw_8694deb3-5198-4f95-934e-4665d56408a9/prometheus-operator-admission-webhook/0.log" Jul 09 04:54:39.926249 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.926211 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx"] Jul 09 04:54:39.926662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.926551 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853f520a-b6d0-4cde-8c82-49ed039cd8b5" containerName="authorino" Jul 09 04:54:39.926662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.926568 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="853f520a-b6d0-4cde-8c82-49ed039cd8b5" containerName="authorino" Jul 09 04:54:39.926662 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.926637 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="853f520a-b6d0-4cde-8c82-49ed039cd8b5" containerName="authorino" Jul 09 04:54:39.929670 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.929648 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:39.931940 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.931918 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vrs45\"/\"default-dockercfg-x6rg7\"" Jul 09 04:54:39.932043 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.932013 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrs45\"/\"openshift-service-ca.crt\"" Jul 09 04:54:39.932973 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.932942 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vrs45\"/\"kube-root-ca.crt\"" Jul 09 04:54:39.938506 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:39.938486 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx"] Jul 09 04:54:40.050386 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.050351 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-proc\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.050386 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.050387 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-podres\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.050636 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.050431 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-lib-modules\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.050636 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.050480 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpqqq\" (UniqueName: \"kubernetes.io/projected/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-kube-api-access-tpqqq\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.050636 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.050567 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-sys\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151406 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151369 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpqqq\" (UniqueName: \"kubernetes.io/projected/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-kube-api-access-tpqqq\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151406 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151422 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-sys\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151450 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-proc\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151468 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-podres\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151500 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-lib-modules\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151552 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-proc\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151552 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-sys\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151655 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-lib-modules\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.151702 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.151661 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-podres\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.158970 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.158948 2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpqqq\" (UniqueName: \"kubernetes.io/projected/c3230b57-f6a6-4e15-83c3-aa0e99d2009f-kube-api-access-tpqqq\") pod \"perf-node-gather-daemonset-8snqx\" (UID: \"c3230b57-f6a6-4e15-83c3-aa0e99d2009f\") " pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.240719 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.240630 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.372510 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.372441 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx"] Jul 09 04:54:40.375299 ip-10-0-130-250 kubenswrapper[2584]: W0709 04:54:40.375275 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc3230b57_f6a6_4e15_83c3_aa0e99d2009f.slice/crio-9dc6d2e4db9dc1070b278003486645f5370fdc150f5be2b2a8dae2df8357400c WatchSource:0}: Error finding container 9dc6d2e4db9dc1070b278003486645f5370fdc150f5be2b2a8dae2df8357400c: Status 404 returned error can't find the container with id 9dc6d2e4db9dc1070b278003486645f5370fdc150f5be2b2a8dae2df8357400c Jul 09 04:54:40.542771 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.542681 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" event={"ID":"c3230b57-f6a6-4e15-83c3-aa0e99d2009f","Type":"ContainerStarted","Data":"c840ed2952f02e4da747279e5472706a67aa42d7c956bde758f5312ca50c4308"} Jul 09 04:54:40.542771 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.542732 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" event={"ID":"c3230b57-f6a6-4e15-83c3-aa0e99d2009f","Type":"ContainerStarted","Data":"9dc6d2e4db9dc1070b278003486645f5370fdc150f5be2b2a8dae2df8357400c"} Jul 09 04:54:40.542945 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.542784 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:40.557254 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:40.557201 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" podStartSLOduration=1.557186867 podStartE2EDuration="1.557186867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 04:54:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 04:54:40.55607029 +0000 UTC m=+2837.793040478" watchObservedRunningTime="2026-07-09 04:54:40.557186867 +0000 UTC m=+2837.794157046" Jul 09 04:54:41.045564 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:41.045516 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c54c65b84-fpnpf_a8de99d4-df02-4c55-85c3-d3b5ea2e00bf/console/0.log" Jul 09 04:54:42.611048 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:42.611006 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2q2pz_66216ebf-6785-49d8-acf3-ba7de5e1ef2c/dns/0.log" Jul 09 04:54:42.631435 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:42.631406 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2q2pz_66216ebf-6785-49d8-acf3-ba7de5e1ef2c/kube-rbac-proxy/0.log" Jul 09 04:54:42.772928 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:42.772893 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bcknn_e5450f12-4d48-4380-b616-623f5b43eaa6/dns-node-resolver/0.log" Jul 09 04:54:43.328515 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:43.328466 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5wl45_b77cdd8d-939d-4119-9384-38c48868d305/node-ca/0.log" Jul 09 04:54:44.287824 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:44.287770 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48c4dm_e357a8ee-eb6e-45bb-80a6-c0aa15966536/istio-proxy/0.log" Jul 09 04:54:44.450947 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:44.450912 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ffb9d954-z4j2p_c025e8a3-7bea-428c-b592-85a126570924/kube-auth-proxy/0.log" Jul 09 04:54:45.316152 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:45.316113 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-78klw_c3d7be2b-3556-4dda-b2fb-d7ed0c822414/serve-healthcheck-canary/0.log" Jul 09 04:54:45.990413 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:45.990386 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t52jt_dba29411-249a-4b04-82ef-7de59d15128d/kube-rbac-proxy/0.log" Jul 09 04:54:46.010775 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:46.010752 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t52jt_dba29411-249a-4b04-82ef-7de59d15128d/exporter/0.log" Jul 09 04:54:46.031278 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:46.031241 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-t52jt_dba29411-249a-4b04-82ef-7de59d15128d/extractor/0.log" Jul 09 04:54:46.556830 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:46.556806 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vrs45/perf-node-gather-daemonset-8snqx" Jul 09 04:54:48.263954 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:48.263926 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fhmnf_a3545f05-462f-4a55-8fba-01b632f438a2/server/0.log" Jul 09 04:54:48.343377 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:48.343341 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-j8cdg_d74231e1-0159-4ab1-8d87-6dee366ea477/manager/0.log" Jul 09 04:54:48.540602 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:48.536264 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-wfxxn_d02ffa15-e098-4367-92fd-0e0c9167df22/manager/0.log" Jul 09 04:54:48.558687 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:48.558646 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cff547d94-wfxxn_d02ffa15-e098-4367-92fd-0e0c9167df22/copy-manifests/0.log" Jul 09 04:54:50.006246 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:50.006219 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-645bc6d555-6fzcj_6f1474e8-f5e7-4f46-b332-a640d86fc6af/manager/0.log" Jul 09 04:54:50.359204 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:50.359119 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48c4dm_e357a8ee-eb6e-45bb-80a6-c0aa15966536/istio-proxy/0.log" Jul 09 04:54:50.474614 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:50.474586 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ffb9d954-z4j2p_c025e8a3-7bea-428c-b592-85a126570924/kube-auth-proxy/0.log" Jul 09 04:54:51.816277 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:51.816180 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf48c4dm_e357a8ee-eb6e-45bb-80a6-c0aa15966536/istio-proxy/0.log" Jul 09 04:54:51.934879 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:51.934841 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-58ffb9d954-z4j2p_c025e8a3-7bea-428c-b592-85a126570924/kube-auth-proxy/0.log" Jul 09 04:54:56.181605 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.181575 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9t6wq_ca4651bf-7ec6-4037-b557-882b89f2fb8b/kube-multus/0.log" Jul 09 04:54:56.360652 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.360617 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wqxhf_457b866c-b844-4825-b6da-19440a29b601/kube-multus-additional-cni-plugins/0.log" Jul 09 04:54:56.388125 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.388091 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wqxhf_457b866c-b844-4825-b6da-19440a29b601/egress-router-binary-copy/0.log" Jul 09 04:54:56.411343 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.411318 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wqxhf_457b866c-b844-4825-b6da-19440a29b601/cni-plugins/0.log" Jul 09 04:54:56.433772 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.433704 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wqxhf_457b866c-b844-4825-b6da-19440a29b601/bond-cni-plugin/0.log" Jul 09 04:54:56.455148 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.455121 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wqxhf_457b866c-b844-4825-b6da-19440a29b601/routeoverride-cni/0.log" Jul 09 04:54:56.476168 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.476142 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wqxhf_457b866c-b844-4825-b6da-19440a29b601/whereabouts-cni-bincopy/0.log" Jul 09 04:54:56.500742 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.497621 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wqxhf_457b866c-b844-4825-b6da-19440a29b601/whereabouts-cni/0.log" Jul 09 04:54:56.808144 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.808056 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vqlpv_65b7ad76-b3e5-48b9-abcc-a556fbc1e905/network-metrics-daemon/0.log" Jul 09 04:54:56.827289 ip-10-0-130-250 kubenswrapper[2584]: I0709 04:54:56.827257 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vqlpv_65b7ad76-b3e5-48b9-abcc-a556fbc1e905/kube-rbac-proxy/0.log"