Jul 10 00:24:09.697075 ip-10-0-131-148 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 10 00:24:09.697084 ip-10-0-131-148 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 10 00:24:09.697091 ip-10-0-131-148 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 10 00:24:09.697292 ip-10-0-131-148 systemd[1]: Failed to start Kubernetes Kubelet. Jul 10 00:24:19.725373 ip-10-0-131-148 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 10 00:24:19.725393 ip-10-0-131-148 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f7dd951a86ea48978968d16d78904967 -- Jul 10 00:26:19.448527 ip-10-0-131-148 systemd[1]: Starting Kubernetes Kubelet... Jul 10 00:26:19.814205 ip-10-0-131-148 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 00:26:19.814205 ip-10-0-131-148 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 00:26:19.814205 ip-10-0-131-148 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 00:26:19.814205 ip-10-0-131-148 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 00:26:19.814205 ip-10-0-131-148 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 00:26:19.815078 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.814990 2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 00:26:19.819284 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819269 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:26:19.819284 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819284 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819288 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819291 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819295 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819298 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819301 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819304 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819307 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819309 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819313 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819316 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819320 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819326 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819329 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819332 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819335 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819338 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819341 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819344 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819347 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:26:19.819352 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819349 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819352 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819355 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819357 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819361 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819364 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819366 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819369 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819372 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819374 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819376 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819379 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819381 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819384 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819386 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819388 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819391 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819393 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819397 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:26:19.819932 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819399 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819410 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819412 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819415 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819417 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819420 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819422 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819425 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819427 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819430 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819432 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819434 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819437 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819440 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819443 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819446 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819448 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819451 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819454 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819456 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:26:19.820403 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819459 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819461 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819463 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819466 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819469 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819471 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819474 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819478 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819481 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819483 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819486 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819490 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819493 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819497 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819500 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819503 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819505 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819508 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819511 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:26:19.820878 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819514 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819516 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819519 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819522 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819525 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819527 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819530 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819533 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819536 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819539 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819542 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819544 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819547 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819549 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819552 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819554 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819556 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819559 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819562 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:26:19.821399 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.819564 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820287 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820293 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820296 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820299 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820301 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820305 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820307 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820310 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820313 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820315 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820318 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820320 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820322 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820325 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820328 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820330 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820333 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820335 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:26:19.821839 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820338 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820342 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820345 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820347 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820351 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820354 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820358 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820360 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820363 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820366 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820368 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820370 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820373 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820376 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820379 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820381 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820384 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820386 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820389 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:26:19.822314 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820391 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820394 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820396 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820399 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820404 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820407 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820410 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820413 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820415 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820418 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820420 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820423 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820425 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820428 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820431 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820433 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820436 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820438 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820441 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:26:19.822769 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820443 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820446 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820449 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820452 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820455 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820458 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820461 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820463 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820466 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820469 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820471 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820473 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820476 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820479 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820481 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820484 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820487 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820489 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820492 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820494 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:26:19.823233 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820497 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820499 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820501 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820504 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820507 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820509 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820512 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820514 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820517 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820519 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820522 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820524 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820527 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820529 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820532 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820534 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820536 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820539 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820541 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820543 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:26:19.823758 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820546 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820548 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820550 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820920 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820925 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820927 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820930 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820933 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820936 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820938 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820941 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820943 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820946 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820949 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820951 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820954 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820956 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820959 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820962 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820964 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:26:19.824258 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820967 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820969 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820972 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820974 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820977 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820979 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820982 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820985 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820987 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820990 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820994 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.820997 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821000 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821003 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821005 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821008 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821010 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821013 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821015 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821017 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:26:19.824746 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821024 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821026 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821029 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821031 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821034 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821036 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821038 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821041 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821043 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821046 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821048 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821050 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821053 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821055 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821057 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821060 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821062 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821065 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821068 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821070 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:26:19.825230 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821073 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821075 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821078 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821080 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821090 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821093 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821096 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821099 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821101 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821105 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821108 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821111 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821116 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821120 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821122 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821124 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821127 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821129 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821132 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:26:19.825757 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821134 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821138 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821140 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821143 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821145 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821148 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821150 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821152 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821155 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821157 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821160 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821164 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821167 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821170 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821173 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821176 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821178 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821181 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821184 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:26:19.826270 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821186 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821189 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821191 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821194 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821280 2579 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821288 2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821297 2579 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821301 2579 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821306 2579 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821309 2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821313 2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821317 2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821321 2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821324 2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821327 2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821331 2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821334 2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821337 2579 flags.go:64] FLAG: --cgroup-root="" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821340 2579 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821342 2579 flags.go:64] FLAG: --client-ca-file="" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821345 2579 flags.go:64] FLAG: --cloud-provider="external" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821348 2579 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821352 2579 flags.go:64] FLAG: --cluster-domain="" Jul 10 00:26:19.826725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821355 2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821358 2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821361 2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821364 2579 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821368 2579 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821371 2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821374 2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821378 2579 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821381 2579 flags.go:64] FLAG: --contention-profiling="false" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821383 2579 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821386 2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821389 2579 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821392 2579 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821396 2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821403 2579 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821408 2579 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821411 2579 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821414 2579 flags.go:64] FLAG: --enable-server="true" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821418 2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821422 2579 flags.go:64] FLAG: --event-burst="100" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821425 2579 flags.go:64] FLAG: --event-qps="50" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821428 2579 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821430 2579 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821433 2579 flags.go:64] FLAG: --eviction-hard="" Jul 10 00:26:19.827297 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821437 2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821439 2579 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821442 2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821445 2579 flags.go:64] FLAG: --eviction-soft="" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821448 2579 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821450 2579 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821453 2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821455 2579 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821458 2579 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821461 2579 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821464 2579 flags.go:64] FLAG: --feature-gates="" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821467 2579 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821470 2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821473 2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821476 2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821478 2579 flags.go:64] FLAG: --healthz-port="10248" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821481 2579 flags.go:64] FLAG: --help="false" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821484 2579 flags.go:64] FLAG: --hostname-override="ip-10-0-131-148.ec2.internal" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821487 2579 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821489 2579 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821492 2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821495 2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821498 2579 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821502 2579 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 00:26:19.827901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821505 2579 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821508 2579 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821511 2579 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821514 2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821517 2579 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821519 2579 flags.go:64] FLAG: --kube-reserved="" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821522 2579 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821524 2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821527 2579 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821530 2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821532 2579 flags.go:64] FLAG: --lock-file="" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821535 2579 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821538 2579 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821541 2579 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821546 2579 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821548 2579 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821551 2579 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821554 2579 flags.go:64] FLAG: --logging-format="text" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821556 2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821560 2579 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821562 2579 flags.go:64] FLAG: --manifest-url="" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821565 2579 flags.go:64] FLAG: --manifest-url-header="" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821570 2579 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821573 2579 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821577 2579 flags.go:64] FLAG: --max-pods="110" Jul 10 00:26:19.828544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821579 2579 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821582 2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821585 2579 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821588 2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821590 2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821593 2579 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821596 2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821603 2579 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821606 2579 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821609 2579 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821613 2579 flags.go:64] FLAG: --pod-cidr="" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821617 2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821622 2579 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821625 2579 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821629 2579 flags.go:64] FLAG: --pods-per-core="0" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821631 2579 flags.go:64] FLAG: --port="10250" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821634 2579 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821637 2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07d6d4f055d7da8af" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821640 2579 flags.go:64] FLAG: --qos-reserved="" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821643 2579 flags.go:64] FLAG: --read-only-port="10255" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821646 2579 flags.go:64] FLAG: --register-node="true" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821648 2579 flags.go:64] FLAG: --register-with-taints="" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821652 2579 flags.go:64] FLAG: --registry-burst="10" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821654 2579 flags.go:64] FLAG: --registry-qps="5" Jul 10 00:26:19.829149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821657 2579 flags.go:64] FLAG: --reserved-cpus="" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821660 2579 flags.go:64] FLAG: --reserved-memory="" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821663 2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821666 2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821669 2579 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821672 2579 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821674 2579 flags.go:64] FLAG: --runonce="false" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821677 2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821680 2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821683 2579 flags.go:64] FLAG: --seccomp-default="false" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821686 2579 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821688 2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821691 2579 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821694 2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821697 2579 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821700 2579 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821703 2579 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821706 2579 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821709 2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821711 2579 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821715 2579 flags.go:64] FLAG: --system-cgroups="" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821717 2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821723 2579 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821726 2579 flags.go:64] FLAG: --tls-cert-file="" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821729 2579 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 00:26:19.829774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821733 2579 flags.go:64] FLAG: --tls-min-version="" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821751 2579 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821755 2579 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821758 2579 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821761 2579 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821764 2579 flags.go:64] FLAG: --v="2" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821770 2579 flags.go:64] FLAG: --version="false" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821774 2579 flags.go:64] FLAG: --vmodule="" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821777 2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.821780 2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821890 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821894 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821898 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821900 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821903 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821905 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821908 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821910 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821912 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821915 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821920 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821923 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:26:19.830389 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821925 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821928 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821930 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821933 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821935 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821938 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821941 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821943 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821947 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821949 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821952 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821954 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821957 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821959 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821962 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821964 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821966 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821969 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821971 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:26:19.830940 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821973 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821976 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821979 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821981 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821983 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821986 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821988 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821990 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821993 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821995 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.821998 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822001 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822005 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822009 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822012 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822015 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822018 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822021 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822024 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822026 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:26:19.831544 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822029 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822032 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822034 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822037 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822039 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822042 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822044 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822047 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822049 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822052 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822054 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822056 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822059 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822061 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822063 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822066 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822068 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822071 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822073 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822075 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:26:19.832356 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822078 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822081 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822084 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822089 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822093 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822096 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822098 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822100 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822103 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822106 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822108 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822111 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822113 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822116 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822119 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822121 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822123 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822126 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822128 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822131 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:26:19.833113 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822133 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822136 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822138 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822140 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822143 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822146 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822150 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.822153 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.822805 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.829211 2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.829225 2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.829261 2579 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.829266 2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829336 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:26:19.833721 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829341 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829345 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829348 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829351 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829353 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829356 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829358 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829361 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829363 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829366 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829368 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829372 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829374 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829377 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829379 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829381 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829384 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829387 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829390 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829392 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:26:19.834063 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829395 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829398 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829400 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829403 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829405 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829408 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829410 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829413 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829416 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829418 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829420 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829423 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829427 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829430 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829433 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829436 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829439 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829441 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829444 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:26:19.834575 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829447 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829449 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829452 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829454 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829458 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829461 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829464 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829466 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829470 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829473 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829476 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829479 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829481 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829484 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829487 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829490 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829493 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829496 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829498 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:26:19.835034 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829501 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829504 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829506 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829509 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829512 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829514 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829517 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829519 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829522 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829524 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829526 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829529 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829531 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829534 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829536 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829539 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829541 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829544 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829546 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829550 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:26:19.835502 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829552 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829555 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829557 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829559 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829562 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829564 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829567 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829570 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829572 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829575 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829577 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829580 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829582 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829585 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829587 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829590 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829592 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829595 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829597 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:26:19.836023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829599 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.829604 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829735 2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829739 2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829743 2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829745 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829748 2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829750 2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829753 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829756 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829758 2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829761 2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829763 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829766 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829769 2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 00:26:19.836527 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829771 2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829774 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829777 2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829780 2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829782 2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829785 2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829787 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829790 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829792 2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829795 2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829797 2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829800 2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829802 2579 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829805 2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829807 2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829810 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829812 2579 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829814 2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829817 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829819 2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 00:26:19.836891 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829822 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829824 2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829826 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829829 2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829831 2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829833 2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829836 2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829838 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829840 2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829843 2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829845 2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829848 2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829850 2579 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829852 2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829855 2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829857 2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829860 2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829862 2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829864 2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829867 2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 00:26:19.837442 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829869 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829871 2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829874 2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829876 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829879 2579 feature_gate.go:328] unrecognized feature gate: Example Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829881 2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829885 2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829888 2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829891 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829894 2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829897 2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829899 2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829901 2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829904 2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829906 2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829909 2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829911 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829915 2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829918 2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 00:26:19.837983 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829921 2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829923 2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829926 2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829928 2579 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829930 2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829933 2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829936 2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829938 2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829940 2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829943 2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829945 2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829947 2579 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829950 2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829952 2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829954 2579 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829957 2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829959 2579 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829962 2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829964 2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829966 2579 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 00:26:19.838547 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829969 2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829971 2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829975 2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829978 2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829981 2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829984 2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:19.829986 2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.829991 2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.830093 2579 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.832821 2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.834106 2579 server.go:1019] "Starting client certificate rotation" Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.834205 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.834232 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 00:26:19.839023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.836448 2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 00:26:19.839379 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.837872 2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 00:26:19.845899 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.845882 2579 log.go:25] "Validated CRI v1 runtime API" Jul 10 00:26:19.847718 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.847702 2579 log.go:25] "Validated CRI v1 image API" Jul 10 00:26:19.849208 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.849188 2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 00:26:19.852295 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.852272 2579 fs.go:135] Filesystem UUIDs: map[09efa2d2-2999-4193-b19a-cfdbcaae98b9:/dev/nvme0n1p3 417e32c4-8eff-4f93-9056-50e2eb93901e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 10 00:26:19.852378 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.852293 2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 00:26:19.857890 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.857774 2579 manager.go:216] Machine: {Timestamp:2026-07-10 00:26:19.856654047 +0000 UTC m=+0.299383089 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098491 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec239bef158e9c5d63c914ff86cb114e SystemUUID:ec239bef-158e-9c5d-63c9-14ff86cb114e BootID:f7dd951a-86ea-4897-8968-d16d78904967 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:63:14:c7:cc:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:63:14:c7:cc:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:23:a2:06:6d:fc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 00:26:19.857890 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.857883 2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 00:26:19.858059 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.858004 2579 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 00:26:19.860519 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.860495 2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 00:26:19.860651 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.860522 2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-148.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 00:26:19.861093 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.861076 2579 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 00:26:19.861129 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.861095 2579 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 00:26:19.861129 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.861103 2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 00:26:19.861661 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.861651 2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 00:26:19.861977 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.861961 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 00:26:19.862798 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.862787 2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 00:26:19.865349 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.865337 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 10 00:26:19.865520 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.865509 2579 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 00:26:19.865559 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.865540 2579 kubelet.go:486] "Attempting to sync node with API server" Jul 10 00:26:19.865559 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.865554 2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 00:26:19.865614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.865569 2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 00:26:19.865614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.865577 2579 kubelet.go:398] "Adding apiserver pod source" Jul 10 00:26:19.865614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.865585 2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 00:26:19.866090 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.866063 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 00:26:19.866168 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.866100 2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 00:26:19.868287 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.868272 2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 00:26:19.870967 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.870950 2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 00:26:19.871037 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.870982 2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 00:26:19.871154 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.871144 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 00:26:19.871209 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.871159 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 00:26:19.871209 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.871166 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 00:26:19.871209 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.871176 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 00:26:19.871209 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.871185 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 00:26:19.871209 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.871191 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 00:26:19.872563 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.872549 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 00:26:19.872599 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.872568 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 00:26:19.872599 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.872578 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 00:26:19.872599 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.872585 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 00:26:19.872679 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.872608 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 00:26:19.872679 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.872617 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 00:26:19.873409 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.873400 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 00:26:19.873442 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.873410 2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 00:26:19.876234 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.876214 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 00:26:19.876331 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.876225 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 00:26:19.876331 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.876300 2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-148.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 00:26:19.876986 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.876972 2579 server.go:1268] "Started kubelet" Jul 10 00:26:19.877079 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.877035 2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 00:26:19.877277 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.877041 2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 00:26:19.877356 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.877303 2579 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 00:26:19.877868 ip-10-0-131-148 systemd[1]: Started Kubernetes Kubelet. Jul 10 00:26:19.878298 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.878180 2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 00:26:19.879507 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.879492 2579 server.go:310] "Adding debug handlers to kubelet server" Jul 10 00:26:19.882186 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.882169 2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 00:26:19.883509 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.883497 2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 00:26:19.883570 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.883511 2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 00:26:19.883744 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.883719 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:19.884048 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884034 2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 00:26:19.884118 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884053 2579 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 00:26:19.884118 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884067 2579 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 00:26:19.884228 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884216 2579 reconstruct.go:97] "Volume reconstruction finished" Jul 10 00:26:19.884228 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884225 2579 reconciler.go:29] "Reconciler: start to sync state" Jul 10 00:26:19.884330 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884318 2579 factory.go:55] Registering systemd factory Jul 10 00:26:19.884386 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884333 2579 factory.go:223] Registration of the systemd container factory successfully Jul 10 00:26:19.884563 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884546 2579 factory.go:153] Registering CRI-O factory Jul 10 00:26:19.884563 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884564 2579 factory.go:223] Registration of the crio container factory successfully Jul 10 00:26:19.884692 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884614 2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 00:26:19.884692 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884638 2579 factory.go:103] Registering Raw factory Jul 10 00:26:19.884692 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.884652 2579 manager.go:1196] Started watching for new ooms in manager Jul 10 00:26:19.885180 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.885165 2579 manager.go:349] Starting recovery of all containers Jul 10 00:26:19.885804 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.884795 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-148.ec2.internal.18c0c6059e9d031a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-148.ec2.internal,UID:ip-10-0-131-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-148.ec2.internal,},FirstTimestamp:2026-07-10 00:26:19.876942618 +0000 UTC m=+0.319671660,LastTimestamp:2026-07-10 00:26:19.876942618 +0000 UTC m=+0.319671660,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-148.ec2.internal,}" Jul 10 00:26:19.888741 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.888711 2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-148.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 00:26:19.888867 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.888848 2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 00:26:19.895303 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.895283 2579 manager.go:354] Recovery completed Jul 10 00:26:19.901040 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.900925 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:19.902808 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.902786 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:19.902876 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.902827 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:19.902876 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.902841 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:19.903422 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.903407 2579 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 00:26:19.903422 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.903421 2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 00:26:19.903552 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.903411 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:19.903552 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.903440 2579 state_mem.go:36] "Initialized new in-memory state store" Jul 10 00:26:19.904193 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.904178 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:19.904243 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.904205 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:19.904243 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.904216 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:19.904663 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.904577 2579 policy_none.go:49] "None policy: Start" Jul 10 00:26:19.904663 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.904593 2579 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 00:26:19.904663 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.904618 2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 00:26:19.905045 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.904980 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-148.ec2.internal.18c0c605a027b89b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-148.ec2.internal,UID:ip-10-0-131-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-148.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-148.ec2.internal,},FirstTimestamp:2026-07-10 00:26:19.902810267 +0000 UTC m=+0.345539318,LastTimestamp:2026-07-10 00:26:19.902810267 +0000 UTC m=+0.345539318,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-148.ec2.internal,}" Jul 10 00:26:19.905593 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.905582 2579 policy_none.go:47] "Start" Jul 10 00:26:19.911664 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.911637 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4qptf" Jul 10 00:26:19.913478 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.913417 2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-148.ec2.internal.18c0c605a0281417 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-148.ec2.internal,UID:ip-10-0-131-148.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-148.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-148.ec2.internal,},FirstTimestamp:2026-07-10 00:26:19.902833687 +0000 UTC m=+0.345562732,LastTimestamp:2026-07-10 00:26:19.902833687 +0000 UTC m=+0.345562732,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-148.ec2.internal,}" Jul 10 00:26:19.918861 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.918841 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4qptf" Jul 10 00:26:19.932879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.932861 2579 manager.go:337] "Starting Device Plugin manager" Jul 10 00:26:19.932986 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.932896 2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 00:26:19.932986 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.932906 2579 server.go:87] "Starting device plugin registration server" Jul 10 00:26:19.933136 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.933124 2579 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 00:26:19.933172 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.933140 2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 00:26:19.933234 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.933218 2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 00:26:19.933327 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.933315 2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 00:26:19.933327 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:19.933325 2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 00:26:19.933905 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.933882 2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 00:26:19.933984 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:19.933932 2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.019039 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.019003 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 00:26:20.021036 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.020302 2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 00:26:20.021036 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.020349 2579 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 00:26:20.021036 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.020370 2579 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 00:26:20.021036 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.020409 2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 00:26:20.022671 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.022645 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:26:20.033595 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.033578 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:20.034532 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.034509 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:20.034634 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.034544 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:20.034634 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.034556 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:20.034634 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.034579 2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.043314 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.043298 2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.043397 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.043320 2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-148.ec2.internal\": node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.066801 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.066735 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.120546 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.120482 2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal"] Jul 10 00:26:20.120668 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.120585 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:20.121537 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.121520 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:20.121636 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.121548 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:20.121636 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.121559 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:20.123800 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.123789 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:20.123927 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.123912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.123966 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.123942 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:20.124490 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.124473 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:20.124549 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.124507 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:20.124549 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.124521 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:20.124549 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.124525 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:20.124549 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.124543 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:20.124675 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.124554 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:20.126619 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.126605 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.126689 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.126629 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:20.127226 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.127213 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:20.127318 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.127239 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:20.127318 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.127268 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:20.144847 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.144822 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-148.ec2.internal\" not found" node="ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.148882 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.148869 2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-148.ec2.internal\" not found" node="ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.166796 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.166776 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.267416 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.267366 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.285702 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.285678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/418d93ee1307ec1e05184ca6752953d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"418d93ee1307ec1e05184ca6752953d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.285769 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.285707 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/418d93ee1307ec1e05184ca6752953d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"418d93ee1307ec1e05184ca6752953d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.285769 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.285726 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24e0180dde756a21daf592d33e7c86a2-config\") pod \"kube-apiserver-proxy-ip-10-0-131-148.ec2.internal\" (UID: \"24e0180dde756a21daf592d33e7c86a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.368053 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.367984 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.386345 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.386325 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/418d93ee1307ec1e05184ca6752953d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"418d93ee1307ec1e05184ca6752953d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.386407 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.386351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24e0180dde756a21daf592d33e7c86a2-config\") pod \"kube-apiserver-proxy-ip-10-0-131-148.ec2.internal\" (UID: \"24e0180dde756a21daf592d33e7c86a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.386407 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.386368 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/418d93ee1307ec1e05184ca6752953d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"418d93ee1307ec1e05184ca6752953d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.386469 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.386415 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/418d93ee1307ec1e05184ca6752953d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"418d93ee1307ec1e05184ca6752953d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.386469 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.386436 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/24e0180dde756a21daf592d33e7c86a2-config\") pod \"kube-apiserver-proxy-ip-10-0-131-148.ec2.internal\" (UID: \"24e0180dde756a21daf592d33e7c86a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.386469 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.386437 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/418d93ee1307ec1e05184ca6752953d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal\" (UID: \"418d93ee1307ec1e05184ca6752953d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.446524 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.446491 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.451270 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.451051 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 10 00:26:20.468534 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.468508 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.569024 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.568986 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.669533 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.669464 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.769975 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.769936 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.833404 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.833378 2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 00:26:20.833890 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.833532 2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 00:26:20.870868 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.870835 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:20.921398 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.921326 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 00:21:20 +0000 UTC" deadline="2027-12-31 22:00:08.500260997 +0000 UTC" Jul 10 00:26:20.921398 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:20.921355 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12957h33m47.578909126s" Jul 10 00:26:20.971598 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:20.971576 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:21.060302 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.060272 2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 00:26:21.062262 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.062228 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientMemory" Jul 10 00:26:21.062365 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.062284 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 00:26:21.062365 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.062300 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeHasSufficientPID" Jul 10 00:26:21.072468 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:21.072449 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:21.150317 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:21.150282 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod418d93ee1307ec1e05184ca6752953d0.slice/crio-c3536c1f65904b709363abe40eec19e1d40a53df5028699333448879916b6a54 WatchSource:0}: Error finding container c3536c1f65904b709363abe40eec19e1d40a53df5028699333448879916b6a54: Status 404 returned error can't find the container with id c3536c1f65904b709363abe40eec19e1d40a53df5028699333448879916b6a54 Jul 10 00:26:21.150918 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:21.150891 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24e0180dde756a21daf592d33e7c86a2.slice/crio-98630b67bb518c63f9ac437129f24f37ea8838372fe990949150e812a4a149a2 WatchSource:0}: Error finding container 98630b67bb518c63f9ac437129f24f37ea8838372fe990949150e812a4a149a2: Status 404 returned error can't find the container with id 98630b67bb518c63f9ac437129f24f37ea8838372fe990949150e812a4a149a2 Jul 10 00:26:21.155204 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.155185 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:26:21.173475 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:21.173430 2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-148.ec2.internal\" not found" Jul 10 00:26:21.227135 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.227107 2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:26:21.230076 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.230059 2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:26:21.284684 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.284650 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" Jul 10 00:26:21.298720 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.298701 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 00:26:21.299707 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.299694 2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" Jul 10 00:26:21.305037 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.305021 2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:26:21.308844 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.308824 2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 00:26:21.866989 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.866933 2579 apiserver.go:52] "Watching apiserver" Jul 10 00:26:21.880560 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.880527 2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 00:26:21.880971 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.880941 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh","openshift-dns/node-resolver-xtbwc","openshift-image-registry/node-ca-wmzlh","openshift-multus/network-metrics-daemon-mqw4x","openshift-network-diagnostics/network-check-target-6k8f8","openshift-network-operator/iptables-alerter-4zsk8","openshift-ovn-kubernetes/ovnkube-node-tjxpd","kube-system/global-pull-secret-syncer-6zh4c","kube-system/konnectivity-agent-hkfgl","kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qbllr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal","openshift-multus/multus-additional-cni-plugins-qfwkw","openshift-multus/multus-m8ldd"] Jul 10 00:26:21.883717 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.883690 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:21.884349 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.884330 2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 00:26:21.886220 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.886074 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.886220 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.886160 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-drcnp\"" Jul 10 00:26:21.886507 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.886489 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 00:26:21.886768 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.886745 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 00:26:21.888411 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.888391 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 00:26:21.888814 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.888793 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-b9778\"" Jul 10 00:26:21.889075 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.889059 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 00:26:21.889299 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.889282 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 00:26:21.889579 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.889560 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 00:26:21.889956 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.889941 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:21.891136 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.891087 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:21.891237 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:21.891163 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:21.891992 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.891970 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 00:26:21.892791 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.892177 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gdfvl\"" Jul 10 00:26:21.892891 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.892663 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 00:26:21.892891 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.892724 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 00:26:21.894967 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.893647 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:21.894967 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:21.893708 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:21.894967 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.894644 2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895659 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-cni-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895691 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-cni-multus\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895719 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-kubelet\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895743 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-conf-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895770 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-os-release\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895795 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/853e3939-f02a-467a-844d-b22ecfd84736-cni-binary-copy\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895819 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-cni-bin\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895855 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-multus-certs\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895880 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3358aa8-4e79-40b8-934a-c892dd63b988-konnectivity-ca\") pod \"konnectivity-agent-hkfgl\" (UID: \"b3358aa8-4e79-40b8-934a-c892dd63b988\") " pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-cnibin\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895927 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-k8s-cni-cncf-io\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-netns\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895974 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-hostroot\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.895999 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/853e3939-f02a-467a-844d-b22ecfd84736-multus-daemon-config\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.896024 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3358aa8-4e79-40b8-934a-c892dd63b988-agent-certs\") pod \"konnectivity-agent-hkfgl\" (UID: \"b3358aa8-4e79-40b8-934a-c892dd63b988\") " pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.896050 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-system-cni-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.896688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.896073 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-socket-dir-parent\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.897356 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.896117 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-etc-kubernetes\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.897356 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.896157 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh962\" (UniqueName: \"kubernetes.io/projected/853e3939-f02a-467a-844d-b22ecfd84736-kube-api-access-wh962\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.897356 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.896439 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.898357 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.898339 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cls26\"" Jul 10 00:26:21.898741 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.898533 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 00:26:21.898741 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.898574 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 00:26:21.899814 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.899098 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.901524 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.901504 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:21.901660 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:21.901638 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:21.902008 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.901989 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 00:26:21.902093 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.902031 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 00:26:21.902155 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.902121 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 00:26:21.902308 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.902273 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 00:26:21.902441 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.902428 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 00:26:21.902488 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.902441 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pfxd2\"" Jul 10 00:26:21.902617 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.902591 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 00:26:21.906119 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.905145 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.906755 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.906734 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.908536 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.908518 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kvs8f\"" Jul 10 00:26:21.908719 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.908626 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 00:26:21.908944 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.908846 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 00:26:21.908944 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.908902 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 00:26:21.909113 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.908957 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 00:26:21.909113 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.909060 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:26:21.909113 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.909069 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hhs5p\"" Jul 10 00:26:21.909960 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.909564 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:21.911634 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.911617 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:26:21.911832 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.911817 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 00:26:21.912022 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.911996 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:21.912173 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.912157 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 00:26:21.912525 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.912507 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8kdfn\"" Jul 10 00:26:21.914196 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.914175 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 00:26:21.914509 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.914488 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 00:26:21.914509 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.914502 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2dvwh\"" Jul 10 00:26:21.921516 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.921490 2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8stl4" Jul 10 00:26:21.930245 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.930222 2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8stl4" Jul 10 00:26:21.985865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.985843 2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 00:26:21.996985 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.996956 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-cnibin\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.997097 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.996998 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.997097 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997028 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysconfig\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.997097 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997047 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysctl-d\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.997097 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997062 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-run\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.997097 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997077 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7886b79d-8489-44af-9072-36341cd95e39-kubelet-config\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:21.997097 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997096 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z25bz\" (UniqueName: \"kubernetes.io/projected/e7299d47-d182-431c-b83d-8a4aa223f2de-kube-api-access-z25bz\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997126 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-kubelet\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997147 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997164 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccd5b99f-444b-455e-b5f3-b18debdf0023-host-slash\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997178 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-system-cni-dir\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997203 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-systemd\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997217 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovn-node-metrics-cert\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997233 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovnkube-script-lib\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997248 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-tmp-dir\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997263 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-os-release\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997302 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/655a9d60-5c3a-455b-832e-56e8a1062561-host\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997327 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-kubelet\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997346 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-run-netns\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997360 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-systemd\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997374 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-cni-bin\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ccd5b99f-444b-455e-b5f3-b18debdf0023-iptables-alerter-script\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997408 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-socket-dir-parent\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997423 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqz2d\" (UniqueName: \"kubernetes.io/projected/655a9d60-5c3a-455b-832e-56e8a1062561-kube-api-access-xqz2d\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997439 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-log-socket\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997454 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bkrw\" (UniqueName: \"kubernetes.io/projected/bbde56ff-c2db-4d64-886b-ff6d2554e16c-kube-api-access-2bkrw\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997471 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-cni-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997492 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-conf-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997507 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-tuned\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997523 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-env-overrides\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997539 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-hosts-file\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997554 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4hjr\" (UniqueName: \"kubernetes.io/projected/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-kube-api-access-l4hjr\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997570 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997593 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-cni-bin\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997611 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-kubernetes\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.997704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997626 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997641 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-etc-selinux\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997657 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-cnibin\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997671 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-slash\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997689 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997704 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-cni-netd\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997725 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997740 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-device-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997756 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3358aa8-4e79-40b8-934a-c892dd63b988-agent-certs\") pod \"konnectivity-agent-hkfgl\" (UID: \"b3358aa8-4e79-40b8-934a-c892dd63b988\") " pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997771 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-modprobe-d\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997789 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-lib-modules\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997806 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997824 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997841 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997859 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-tmp\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997874 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/655a9d60-5c3a-455b-832e-56e8a1062561-serviceca\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:21.998215 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997889 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovnkube-config\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997904 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjd86\" (UniqueName: \"kubernetes.io/projected/12ba7873-dbe9-43a9-9f47-86f4280640c6-kube-api-access-jjd86\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997920 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-socket-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997935 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-registration-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997950 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-var-lib-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997966 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997982 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-hostroot\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.997997 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-host\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998013 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-system-cni-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998029 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-etc-kubernetes\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998052 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh962\" (UniqueName: \"kubernetes.io/projected/853e3939-f02a-467a-844d-b22ecfd84736-kube-api-access-wh962\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998068 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-var-lib-kubelet\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998084 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vl867\" (UniqueName: \"kubernetes.io/projected/d5f71bf8-0d90-4b94-84c2-e706db351d5a-kube-api-access-vl867\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998100 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-sys-fs\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998117 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-cni-multus\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998132 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:21.998696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998148 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-systemd-units\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998167 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7886b79d-8489-44af-9072-36341cd95e39-dbus\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998186 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjr42\" (UniqueName: \"kubernetes.io/projected/ccd5b99f-444b-455e-b5f3-b18debdf0023-kube-api-access-fjr42\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998203 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-os-release\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998219 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/853e3939-f02a-467a-844d-b22ecfd84736-cni-binary-copy\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998235 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-multus-certs\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998251 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysctl-conf\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998266 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-sys\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998282 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-etc-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998299 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-ovn\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998314 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-node-log\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998353 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3358aa8-4e79-40b8-934a-c892dd63b988-konnectivity-ca\") pod \"konnectivity-agent-hkfgl\" (UID: \"b3358aa8-4e79-40b8-934a-c892dd63b988\") " pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-k8s-cni-cncf-io\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-netns\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998538 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-multus-certs\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998550 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/853e3939-f02a-467a-844d-b22ecfd84736-multus-daemon-config\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998564 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-k8s-cni-cncf-io\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:21.999312 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998580 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmdb8\" (UniqueName: \"kubernetes.io/projected/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-kube-api-access-wmdb8\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998596 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-system-cni-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998633 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-kubelet\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998641 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-cni-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998647 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-cni-bin\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998694 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-conf-dir\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998729 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-os-release\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998738 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-run-netns\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998800 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-cnibin\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998806 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-multus-socket-dir-parent\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998843 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-etc-kubernetes\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.998845 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-hostroot\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.999131 2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.999267 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b3358aa8-4e79-40b8-934a-c892dd63b988-konnectivity-ca\") pod \"konnectivity-agent-hkfgl\" (UID: \"b3358aa8-4e79-40b8-934a-c892dd63b988\") " pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.999403 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/853e3939-f02a-467a-844d-b22ecfd84736-cni-binary-copy\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.999424 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/853e3939-f02a-467a-844d-b22ecfd84736-multus-daemon-config\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.000178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:21.999476 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/853e3939-f02a-467a-844d-b22ecfd84736-host-var-lib-cni-multus\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.003129 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.003105 2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 00:26:22.004175 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.004149 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b3358aa8-4e79-40b8-934a-c892dd63b988-agent-certs\") pod \"konnectivity-agent-hkfgl\" (UID: \"b3358aa8-4e79-40b8-934a-c892dd63b988\") " pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:22.009913 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.009620 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh962\" (UniqueName: \"kubernetes.io/projected/853e3939-f02a-467a-844d-b22ecfd84736-kube-api-access-wh962\") pod \"multus-m8ldd\" (UID: \"853e3939-f02a-467a-844d-b22ecfd84736\") " pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.026488 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.026446 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" event={"ID":"418d93ee1307ec1e05184ca6752953d0","Type":"ContainerStarted","Data":"c3536c1f65904b709363abe40eec19e1d40a53df5028699333448879916b6a54"} Jul 10 00:26:22.028980 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.028953 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" event={"ID":"24e0180dde756a21daf592d33e7c86a2","Type":"ContainerStarted","Data":"98630b67bb518c63f9ac437129f24f37ea8838372fe990949150e812a4a149a2"} Jul 10 00:26:22.099414 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099381 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccd5b99f-444b-455e-b5f3-b18debdf0023-host-slash\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:22.099580 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099441 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-system-cni-dir\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.099580 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099463 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ccd5b99f-444b-455e-b5f3-b18debdf0023-host-slash\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:22.099580 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-systemd\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.099580 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099516 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovn-node-metrics-cert\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.099580 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099536 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-systemd\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.099580 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099539 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-system-cni-dir\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.099580 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099543 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovnkube-script-lib\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099582 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-tmp-dir\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099611 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-os-release\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099636 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/655a9d60-5c3a-455b-832e-56e8a1062561-host\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099659 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-kubelet\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-run-netns\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099708 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-systemd\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-cni-bin\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ccd5b99f-444b-455e-b5f3-b18debdf0023-iptables-alerter-script\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099786 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqz2d\" (UniqueName: \"kubernetes.io/projected/655a9d60-5c3a-455b-832e-56e8a1062561-kube-api-access-xqz2d\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099810 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-log-socket\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.099879 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099834 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bkrw\" (UniqueName: \"kubernetes.io/projected/bbde56ff-c2db-4d64-886b-ff6d2554e16c-kube-api-access-2bkrw\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099904 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-tuned\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099917 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-run-netns\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099931 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-env-overrides\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.099975 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-hosts-file\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100007 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4hjr\" (UniqueName: \"kubernetes.io/projected/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-kube-api-access-l4hjr\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100035 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100067 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-kubernetes\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100093 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100122 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-etc-selinux\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100147 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-slash\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100158 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovnkube-script-lib\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100173 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100198 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-cni-netd\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100225 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100250 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-device-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100286 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-env-overrides\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.100506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100301 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-modprobe-d\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100326 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-lib-modules\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100354 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100383 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100410 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100424 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-cni-bin\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100439 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-tmp\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/655a9d60-5c3a-455b-832e-56e8a1062561-serviceca\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100491 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovnkube-config\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100518 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjd86\" (UniqueName: \"kubernetes.io/projected/12ba7873-dbe9-43a9-9f47-86f4280640c6-kube-api-access-jjd86\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100543 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-socket-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100569 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-registration-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100594 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-var-lib-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100621 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100646 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-host\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100676 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-var-lib-kubelet\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100682 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-tmp-dir\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:22.101487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100705 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vl867\" (UniqueName: \"kubernetes.io/projected/d5f71bf8-0d90-4b94-84c2-e706db351d5a-kube-api-access-vl867\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100734 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-sys-fs\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100765 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100791 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-systemd-units\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100790 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-os-release\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100846 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/655a9d60-5c3a-455b-832e-56e8a1062561-host\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100880 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7886b79d-8489-44af-9072-36341cd95e39-dbus\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100908 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjr42\" (UniqueName: \"kubernetes.io/projected/ccd5b99f-444b-455e-b5f3-b18debdf0023-kube-api-access-fjr42\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100942 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysctl-conf\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-sys\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100995 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-etc-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101018 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-ovn\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101041 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-node-log\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101074 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmdb8\" (UniqueName: \"kubernetes.io/projected/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-kube-api-access-wmdb8\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101087 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-var-lib-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101100 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-cnibin\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101136 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-cnibin\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.102346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101137 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101174 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysconfig\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101211 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysctl-d\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101237 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-run\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101262 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7886b79d-8489-44af-9072-36341cd95e39-kubelet-config\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101289 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z25bz\" (UniqueName: \"kubernetes.io/projected/e7299d47-d182-431c-b83d-8a4aa223f2de-kube-api-access-z25bz\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101336 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-kubelet\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101347 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-device-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101340 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101400 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101476 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-host\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101481 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-hosts-file\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101482 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysctl-conf\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-log-socket\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101571 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-node-log\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101626 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101717 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-modprobe-d\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103184 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101764 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-var-lib-kubelet\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101777 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-systemd-units\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.100384 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-systemd\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101804 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-etc-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101801 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/12ba7873-dbe9-43a9-9f47-86f4280640c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101825 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-registration-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101892 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysconfig\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101897 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7886b79d-8489-44af-9072-36341cd95e39-kubelet-config\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101907 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-socket-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101939 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-slash\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101943 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-sysctl-d\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101950 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101955 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-ovn\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101722 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-sys\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.103943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.102006 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-sys-fs\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.104611 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.101978 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-lib-modules\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.106560 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.106525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovnkube-config\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.106654 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.106620 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e7299d47-d182-431c-b83d-8a4aa223f2de-etc-selinux\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.107223 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.107200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/655a9d60-5c3a-455b-832e-56e8a1062561-serviceca\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:22.107371 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.107354 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7886b79d-8489-44af-9072-36341cd95e39-dbus\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:22.107435 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.107410 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-run\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.107503 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.107469 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:22.107586 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.107571 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:26:22.60752858 +0000 UTC m=+3.050257635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:22.108065 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.108046 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ccd5b99f-444b-455e-b5f3-b18debdf0023-iptables-alerter-script\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:22.108130 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.108118 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:22.108184 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.108174 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret podName:7886b79d-8489-44af-9072-36341cd95e39 nodeName:}" failed. No retries permitted until 2026-07-10 00:26:22.608152387 +0000 UTC m=+3.050881437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret") pod "global-pull-secret-syncer-6zh4c" (UID: "7886b79d-8489-44af-9072-36341cd95e39") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:22.108382 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.108365 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-tuned\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.108450 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.108434 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-etc-kubernetes\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.108814 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.108797 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bbde56ff-c2db-4d64-886b-ff6d2554e16c-ovn-node-metrics-cert\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.109356 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.102007 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-run-ovn-kubernetes\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.110739 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.110713 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/12ba7873-dbe9-43a9-9f47-86f4280640c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.110839 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.110804 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-run-openvswitch\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.113483 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.112620 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:26:22.113483 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.112646 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:26:22.113483 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.112662 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fl7vv for pod openshift-network-diagnostics/network-check-target-6k8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:22.113483 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.112735 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv podName:ddd92337-9f34-40a2-8c24-9049b923c61e nodeName:}" failed. No retries permitted until 2026-07-10 00:26:22.612717873 +0000 UTC m=+3.055446916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fl7vv" (UniqueName: "kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv") pod "network-check-target-6k8f8" (UID: "ddd92337-9f34-40a2-8c24-9049b923c61e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:22.113483 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.102003 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.113483 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.113456 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bbde56ff-c2db-4d64-886b-ff6d2554e16c-host-cni-netd\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.113940 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.113914 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4hjr\" (UniqueName: \"kubernetes.io/projected/07492d99-1fb7-4079-ac5a-1c9e41d1ed5c-kube-api-access-l4hjr\") pod \"node-resolver-xtbwc\" (UID: \"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c\") " pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:22.117572 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.117223 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqz2d\" (UniqueName: \"kubernetes.io/projected/655a9d60-5c3a-455b-832e-56e8a1062561-kube-api-access-xqz2d\") pod \"node-ca-wmzlh\" (UID: \"655a9d60-5c3a-455b-832e-56e8a1062561\") " pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:22.117729 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.117709 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z25bz\" (UniqueName: \"kubernetes.io/projected/e7299d47-d182-431c-b83d-8a4aa223f2de-kube-api-access-z25bz\") pod \"aws-ebs-csi-driver-node-zm6mh\" (UID: \"e7299d47-d182-431c-b83d-8a4aa223f2de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.117945 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.117920 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjr42\" (UniqueName: \"kubernetes.io/projected/ccd5b99f-444b-455e-b5f3-b18debdf0023-kube-api-access-fjr42\") pod \"iptables-alerter-4zsk8\" (UID: \"ccd5b99f-444b-455e-b5f3-b18debdf0023\") " pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:22.118267 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.118243 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-tmp\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.118486 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.118442 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjd86\" (UniqueName: \"kubernetes.io/projected/12ba7873-dbe9-43a9-9f47-86f4280640c6-kube-api-access-jjd86\") pod \"multus-additional-cni-plugins-qfwkw\" (UID: \"12ba7873-dbe9-43a9-9f47-86f4280640c6\") " pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.118660 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.118640 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmdb8\" (UniqueName: \"kubernetes.io/projected/094d74e4-7295-40a4-8f78-5d70b4d3a6d6-kube-api-access-wmdb8\") pod \"tuned-qbllr\" (UID: \"094d74e4-7295-40a4-8f78-5d70b4d3a6d6\") " pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.118829 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.118804 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bkrw\" (UniqueName: \"kubernetes.io/projected/bbde56ff-c2db-4d64-886b-ff6d2554e16c-kube-api-access-2bkrw\") pod \"ovnkube-node-tjxpd\" (UID: \"bbde56ff-c2db-4d64-886b-ff6d2554e16c\") " pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.123775 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.123754 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vl867\" (UniqueName: \"kubernetes.io/projected/d5f71bf8-0d90-4b94-84c2-e706db351d5a-kube-api-access-vl867\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:22.197894 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.197857 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:22.208874 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.208684 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m8ldd" Jul 10 00:26:22.211013 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.210978 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3358aa8_4e79_40b8_934a_c892dd63b988.slice/crio-14b6caf4a75603f9de90b420cbfd0f6c69ce9a43c66c09a886b11b8c8cce418b WatchSource:0}: Error finding container 14b6caf4a75603f9de90b420cbfd0f6c69ce9a43c66c09a886b11b8c8cce418b: Status 404 returned error can't find the container with id 14b6caf4a75603f9de90b420cbfd0f6c69ce9a43c66c09a886b11b8c8cce418b Jul 10 00:26:22.219588 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.219561 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853e3939_f02a_467a_844d_b22ecfd84736.slice/crio-08c58a86f4ae0d93b9dd57c9f869957c00ae87e21c16c04fb9915a79676d464d WatchSource:0}: Error finding container 08c58a86f4ae0d93b9dd57c9f869957c00ae87e21c16c04fb9915a79676d464d: Status 404 returned error can't find the container with id 08c58a86f4ae0d93b9dd57c9f869957c00ae87e21c16c04fb9915a79676d464d Jul 10 00:26:22.220570 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.220550 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:22.230686 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.230661 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbde56ff_c2db_4d64_886b_ff6d2554e16c.slice/crio-bcd4ada8f9d8b3adab818d7d9b1da026fef57bab0c916424c062a0d4facdc9cb WatchSource:0}: Error finding container bcd4ada8f9d8b3adab818d7d9b1da026fef57bab0c916424c062a0d4facdc9cb: Status 404 returned error can't find the container with id bcd4ada8f9d8b3adab818d7d9b1da026fef57bab0c916424c062a0d4facdc9cb Jul 10 00:26:22.231789 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.231759 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmzlh" Jul 10 00:26:22.237152 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.237130 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" Jul 10 00:26:22.242451 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.242426 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655a9d60_5c3a_455b_832e_56e8a1062561.slice/crio-81399ca02838c8338da6d5f63143b5edd4589e9117a7fa569fb3dea8ac6e92b0 WatchSource:0}: Error finding container 81399ca02838c8338da6d5f63143b5edd4589e9117a7fa569fb3dea8ac6e92b0: Status 404 returned error can't find the container with id 81399ca02838c8338da6d5f63143b5edd4589e9117a7fa569fb3dea8ac6e92b0 Jul 10 00:26:22.247250 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.247229 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" Jul 10 00:26:22.250920 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.250896 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12ba7873_dbe9_43a9_9f47_86f4280640c6.slice/crio-2dd830a4288645827d334562d0f27c48a302b21666863cd6143f0b525812e86e WatchSource:0}: Error finding container 2dd830a4288645827d334562d0f27c48a302b21666863cd6143f0b525812e86e: Status 404 returned error can't find the container with id 2dd830a4288645827d334562d0f27c48a302b21666863cd6143f0b525812e86e Jul 10 00:26:22.255378 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.255356 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qbllr" Jul 10 00:26:22.258560 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.258537 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7299d47_d182_431c_b83d_8a4aa223f2de.slice/crio-3d3e8a8026b31d4e09439d0b2907372f06fc19b02510acb328ef77e6fc51b17f WatchSource:0}: Error finding container 3d3e8a8026b31d4e09439d0b2907372f06fc19b02510acb328ef77e6fc51b17f: Status 404 returned error can't find the container with id 3d3e8a8026b31d4e09439d0b2907372f06fc19b02510acb328ef77e6fc51b17f Jul 10 00:26:22.264203 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.264184 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4zsk8" Jul 10 00:26:22.265150 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.265127 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094d74e4_7295_40a4_8f78_5d70b4d3a6d6.slice/crio-137bb907745f433b739757dfb908778f3102b15b1a39e3af87df3d51cbd8afae WatchSource:0}: Error finding container 137bb907745f433b739757dfb908778f3102b15b1a39e3af87df3d51cbd8afae: Status 404 returned error can't find the container with id 137bb907745f433b739757dfb908778f3102b15b1a39e3af87df3d51cbd8afae Jul 10 00:26:22.268728 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.268707 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xtbwc" Jul 10 00:26:22.275379 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.275354 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd5b99f_444b_455e_b5f3_b18debdf0023.slice/crio-5679435ab89520e1bbabce8e966c79b140186e6ea805e0c057fc9ad1220eb042 WatchSource:0}: Error finding container 5679435ab89520e1bbabce8e966c79b140186e6ea805e0c057fc9ad1220eb042: Status 404 returned error can't find the container with id 5679435ab89520e1bbabce8e966c79b140186e6ea805e0c057fc9ad1220eb042 Jul 10 00:26:22.280985 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:26:22.280959 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07492d99_1fb7_4079_ac5a_1c9e41d1ed5c.slice/crio-4a84160daf6211c3d740440b437a53ef303e00d6b3133543924cdc4170f9ff2f WatchSource:0}: Error finding container 4a84160daf6211c3d740440b437a53ef303e00d6b3133543924cdc4170f9ff2f: Status 404 returned error can't find the container with id 4a84160daf6211c3d740440b437a53ef303e00d6b3133543924cdc4170f9ff2f Jul 10 00:26:22.706559 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.706529 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:22.706725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.706567 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:22.706725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.706601 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:22.706725 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706715 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:22.706725 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706725 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:26:22.706958 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706746 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:26:22.706958 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706757 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fl7vv for pod openshift-network-diagnostics/network-check-target-6k8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:22.706958 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706715 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:22.706958 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706784 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:26:23.706762629 +0000 UTC m=+4.149491665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:22.706958 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706826 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv podName:ddd92337-9f34-40a2-8c24-9049b923c61e nodeName:}" failed. No retries permitted until 2026-07-10 00:26:23.706812663 +0000 UTC m=+4.149541693 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fl7vv" (UniqueName: "kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv") pod "network-check-target-6k8f8" (UID: "ddd92337-9f34-40a2-8c24-9049b923c61e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:22.706958 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:22.706844 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret podName:7886b79d-8489-44af-9072-36341cd95e39 nodeName:}" failed. No retries permitted until 2026-07-10 00:26:23.706833094 +0000 UTC m=+4.149562126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret") pod "global-pull-secret-syncer-6zh4c" (UID: "7886b79d-8489-44af-9072-36341cd95e39") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:22.935862 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.935784 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:21:22 +0000 UTC" deadline="2027-12-22 00:16:52.402377374 +0000 UTC" Jul 10 00:26:22.935862 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:22.935865 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12719h50m29.466523766s" Jul 10 00:26:23.021017 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.020933 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:23.021174 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.021079 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:23.033218 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.033181 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4zsk8" event={"ID":"ccd5b99f-444b-455e-b5f3-b18debdf0023","Type":"ContainerStarted","Data":"5679435ab89520e1bbabce8e966c79b140186e6ea805e0c057fc9ad1220eb042"} Jul 10 00:26:23.034858 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.034825 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" event={"ID":"e7299d47-d182-431c-b83d-8a4aa223f2de","Type":"ContainerStarted","Data":"3d3e8a8026b31d4e09439d0b2907372f06fc19b02510acb328ef77e6fc51b17f"} Jul 10 00:26:23.036389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.036356 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerStarted","Data":"2dd830a4288645827d334562d0f27c48a302b21666863cd6143f0b525812e86e"} Jul 10 00:26:23.038042 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.038014 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"bcd4ada8f9d8b3adab818d7d9b1da026fef57bab0c916424c062a0d4facdc9cb"} Jul 10 00:26:23.039542 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.039520 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xtbwc" event={"ID":"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c","Type":"ContainerStarted","Data":"4a84160daf6211c3d740440b437a53ef303e00d6b3133543924cdc4170f9ff2f"} Jul 10 00:26:23.040901 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.040876 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbllr" event={"ID":"094d74e4-7295-40a4-8f78-5d70b4d3a6d6","Type":"ContainerStarted","Data":"137bb907745f433b739757dfb908778f3102b15b1a39e3af87df3d51cbd8afae"} Jul 10 00:26:23.041968 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.041948 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmzlh" event={"ID":"655a9d60-5c3a-455b-832e-56e8a1062561","Type":"ContainerStarted","Data":"81399ca02838c8338da6d5f63143b5edd4589e9117a7fa569fb3dea8ac6e92b0"} Jul 10 00:26:23.043226 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.043202 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m8ldd" event={"ID":"853e3939-f02a-467a-844d-b22ecfd84736","Type":"ContainerStarted","Data":"08c58a86f4ae0d93b9dd57c9f869957c00ae87e21c16c04fb9915a79676d464d"} Jul 10 00:26:23.044150 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.044127 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkfgl" event={"ID":"b3358aa8-4e79-40b8-934a-c892dd63b988","Type":"ContainerStarted","Data":"14b6caf4a75603f9de90b420cbfd0f6c69ce9a43c66c09a886b11b8c8cce418b"} Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.713506 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.713766 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.713795 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.713810 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fl7vv for pod openshift-network-diagnostics/network-check-target-6k8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.713860 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.713873 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv podName:ddd92337-9f34-40a2-8c24-9049b923c61e nodeName:}" failed. No retries permitted until 2026-07-10 00:26:25.713851634 +0000 UTC m=+6.156580686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fl7vv" (UniqueName: "kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv") pod "network-check-target-6k8f8" (UID: "ddd92337-9f34-40a2-8c24-9049b923c61e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.713785 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.713916 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.713991 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.714028 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:26:25.714016832 +0000 UTC m=+6.156745863 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:23.714150 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:23.714051 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret podName:7886b79d-8489-44af-9072-36341cd95e39 nodeName:}" failed. No retries permitted until 2026-07-10 00:26:25.714041367 +0000 UTC m=+6.156770398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret") pod "global-pull-secret-syncer-6zh4c" (UID: "7886b79d-8489-44af-9072-36341cd95e39") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:23.936761 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.936691 2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 00:21:22 +0000 UTC" deadline="2028-02-27 04:00:39.521183313 +0000 UTC" Jul 10 00:26:23.936761 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:23.936728 2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14331h34m15.584460458s" Jul 10 00:26:24.023106 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:24.022734 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:24.023106 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:24.022936 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:24.023452 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:24.023429 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:24.023574 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:24.023539 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:24.053988 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:24.053959 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" event={"ID":"24e0180dde756a21daf592d33e7c86a2","Type":"ContainerStarted","Data":"aaae1a0105eb461685be6c042e14fa880b482998f5ae2c51237fe1305ffee81e"} Jul 10 00:26:24.068884 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:24.068792 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-148.ec2.internal" podStartSLOduration=2.068769595 podStartE2EDuration="2.068769595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:24.068290236 +0000 UTC m=+4.511019296" watchObservedRunningTime="2026-07-10 00:26:24.068769595 +0000 UTC m=+4.511498652" Jul 10 00:26:25.021555 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:25.021492 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:25.022054 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.021665 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:25.063523 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:25.063487 2579 generic.go:356] "Generic (PLEG): container finished" podID="418d93ee1307ec1e05184ca6752953d0" containerID="9466f707b67ade762b104a43bac9cdce54f6d227328e6c51b03f2da8b5ce5cd8" exitCode=0 Jul 10 00:26:25.063757 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:25.063583 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" event={"ID":"418d93ee1307ec1e05184ca6752953d0","Type":"ContainerDied","Data":"9466f707b67ade762b104a43bac9cdce54f6d227328e6c51b03f2da8b5ce5cd8"} Jul 10 00:26:25.732976 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:25.732935 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:25.732976 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:25.732995 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:25.733237 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:25.733037 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:25.733237 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733111 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:25.733237 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733185 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret podName:7886b79d-8489-44af-9072-36341cd95e39 nodeName:}" failed. No retries permitted until 2026-07-10 00:26:29.733167583 +0000 UTC m=+10.175896614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret") pod "global-pull-secret-syncer-6zh4c" (UID: "7886b79d-8489-44af-9072-36341cd95e39") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:25.733237 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733213 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:26:25.733237 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733237 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:26:25.733529 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733250 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fl7vv for pod openshift-network-diagnostics/network-check-target-6k8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:25.733529 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733300 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv podName:ddd92337-9f34-40a2-8c24-9049b923c61e nodeName:}" failed. No retries permitted until 2026-07-10 00:26:29.733285136 +0000 UTC m=+10.176014172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fl7vv" (UniqueName: "kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv") pod "network-check-target-6k8f8" (UID: "ddd92337-9f34-40a2-8c24-9049b923c61e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:25.733529 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733389 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:25.733529 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:25.733481 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:26:29.73346577 +0000 UTC m=+10.176194801 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:26.021303 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:26.021220 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:26.021496 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:26.021399 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:26.021834 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:26.021810 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:26.022225 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:26.021931 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:27.020851 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:27.020815 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:27.021064 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:27.020959 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:28.022979 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:28.022899 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:28.023474 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:28.023003 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:28.023474 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:28.023059 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:28.023474 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:28.023114 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:29.020577 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:29.020540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:29.020777 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.020674 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:29.768569 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:29.768473 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:29.768569 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:29.768526 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:29.768569 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:29.768570 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:29.769092 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.768741 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:26:29.769092 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.768766 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:26:29.769092 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.768780 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fl7vv for pod openshift-network-diagnostics/network-check-target-6k8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:29.769092 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.768829 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv podName:ddd92337-9f34-40a2-8c24-9049b923c61e nodeName:}" failed. No retries permitted until 2026-07-10 00:26:37.768814765 +0000 UTC m=+18.211543796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fl7vv" (UniqueName: "kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv") pod "network-check-target-6k8f8" (UID: "ddd92337-9f34-40a2-8c24-9049b923c61e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:29.769255 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.769243 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:29.769343 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.769296 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret podName:7886b79d-8489-44af-9072-36341cd95e39 nodeName:}" failed. No retries permitted until 2026-07-10 00:26:37.769279809 +0000 UTC m=+18.212008864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret") pod "global-pull-secret-syncer-6zh4c" (UID: "7886b79d-8489-44af-9072-36341cd95e39") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:29.769410 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.769394 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:29.769453 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:29.769428 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:26:37.769419889 +0000 UTC m=+18.212148920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:30.021977 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:30.021886 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:30.022140 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:30.022003 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:30.022450 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:30.022427 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:30.022599 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:30.022556 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:31.021460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:31.021381 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:31.021885 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:31.021511 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:32.021180 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:32.021146 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:32.021335 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:32.021275 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:32.021370 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:32.021278 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:32.021416 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:32.021362 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:33.020875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:33.020831 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:33.021404 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:33.020959 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:34.021083 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.020910 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:34.021516 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.020905 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:34.021516 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:34.021192 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:34.021516 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:34.021288 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:34.086010 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.085975 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" event={"ID":"418d93ee1307ec1e05184ca6752953d0","Type":"ContainerStarted","Data":"27cc6800a79b462523a1236345bffd9e4ee654deb34962fcf7863070593d4823"} Jul 10 00:26:34.087682 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.087653 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" event={"ID":"e7299d47-d182-431c-b83d-8a4aa223f2de","Type":"ContainerStarted","Data":"9d2062f884c9fe16160e7fe5193587568620c1adac4b44c968b184baf78b3418"} Jul 10 00:26:34.089535 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.089510 2579 generic.go:356] "Generic (PLEG): container finished" podID="12ba7873-dbe9-43a9-9f47-86f4280640c6" containerID="efb2cd9f5c215515a69a39da45014c2a90f15258845953c308901a660d8a2a6d" exitCode=0 Jul 10 00:26:34.089617 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.089600 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerDied","Data":"efb2cd9f5c215515a69a39da45014c2a90f15258845953c308901a660d8a2a6d"} Jul 10 00:26:34.091846 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.091821 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xtbwc" event={"ID":"07492d99-1fb7-4079-ac5a-1c9e41d1ed5c","Type":"ContainerStarted","Data":"3a16486975a4ba0a594075e4b82ca7ebf8e5017c4f894a89ea9e3eba7bf5f38e"} Jul 10 00:26:34.093521 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.093496 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qbllr" event={"ID":"094d74e4-7295-40a4-8f78-5d70b4d3a6d6","Type":"ContainerStarted","Data":"95ca2b5cb3bd9e733f78d9fc0045df11536306eb89f690a090d74b1aea088938"} Jul 10 00:26:34.096594 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.096563 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmzlh" event={"ID":"655a9d60-5c3a-455b-832e-56e8a1062561","Type":"ContainerStarted","Data":"0e0d72b93cb63754bea020abfd9b31168ff39016775494234b9f96f1ece1e49d"} Jul 10 00:26:34.099025 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.098982 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-148.ec2.internal" podStartSLOduration=12.098967178 podStartE2EDuration="12.098967178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:34.098808022 +0000 UTC m=+14.541537073" watchObservedRunningTime="2026-07-10 00:26:34.098967178 +0000 UTC m=+14.541696230" Jul 10 00:26:34.099748 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.099576 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hkfgl" event={"ID":"b3358aa8-4e79-40b8-934a-c892dd63b988","Type":"ContainerStarted","Data":"eb36dadbc646a608f52323f4e0c4c074c61849d19ba6dca02d6cc837a5189209"} Jul 10 00:26:34.112099 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.112053 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qbllr" podStartSLOduration=1.6764262859999999 podStartE2EDuration="13.112037522s" totalImagesPullingTime="11.435611236s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:34.112004329 +0000 UTC m=+14.554733382" watchObservedRunningTime="2026-07-10 00:26:34.112037522 +0000 UTC m=+14.554766574" Jul 10 00:26:34.125408 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.125326 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xtbwc" podStartSLOduration=2.047816525 podStartE2EDuration="13.125313249s" totalImagesPullingTime="11.077496724s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:34.124695915 +0000 UTC m=+14.567424967" watchObservedRunningTime="2026-07-10 00:26:34.125313249 +0000 UTC m=+14.568042300" Jul 10 00:26:34.138895 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.138847 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wmzlh" podStartSLOduration=2.9661962429999997 podStartE2EDuration="14.138830143s" totalImagesPullingTime="11.1726339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:34.138260061 +0000 UTC m=+14.580989104" watchObservedRunningTime="2026-07-10 00:26:34.138830143 +0000 UTC m=+14.581559194" Jul 10 00:26:34.181784 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:34.181729 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hkfgl" podStartSLOduration=2.989584433 podStartE2EDuration="14.181708894s" totalImagesPullingTime="11.192124461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:34.181663713 +0000 UTC m=+14.624392763" watchObservedRunningTime="2026-07-10 00:26:34.181708894 +0000 UTC m=+14.624437947" Jul 10 00:26:35.006545 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.006499 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:35.007135 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.007105 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:35.021088 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.021056 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:35.021570 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:35.021242 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:35.103757 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.103719 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4zsk8" event={"ID":"ccd5b99f-444b-455e-b5f3-b18debdf0023","Type":"ContainerStarted","Data":"486b39773db7937985e6d2d283d19b325ff30c3d251b0677ffadad55c0e11653"} Jul 10 00:26:35.120463 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.120415 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4zsk8" podStartSLOduration=3.038753926 podStartE2EDuration="14.120399874s" totalImagesPullingTime="11.081645948s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:35.12011864 +0000 UTC m=+15.562847691" watchObservedRunningTime="2026-07-10 00:26:35.120399874 +0000 UTC m=+15.563128924" Jul 10 00:26:35.192472 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.192387 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:35.192929 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.192909 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hkfgl" Jul 10 00:26:35.405059 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.405035 2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 00:26:35.939544 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.939425 2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T00:26:35.405054539Z","UUID":"ad86fc43-8164-4bcc-906d-abeec3bf7f20","Handler":null,"Name":"","Endpoint":""} Jul 10 00:26:35.941157 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.941134 2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 00:26:35.941157 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:35.941177 2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 00:26:36.021136 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:36.021104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:36.021764 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:36.021104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:36.021764 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:36.021223 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:36.021764 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:36.021304 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:36.110308 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:36.110275 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" event={"ID":"e7299d47-d182-431c-b83d-8a4aa223f2de","Type":"ContainerStarted","Data":"571d6b224678dbecb7e27b6553a4e37cd94599e04ba73fdf1e0d179e3621ee18"} Jul 10 00:26:37.021555 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:37.021524 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:37.022118 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.021635 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:37.831188 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:37.831149 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:37.831188 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:37.831194 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:37.831222 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831298 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831330 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831358 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831385 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret podName:7886b79d-8489-44af-9072-36341cd95e39 nodeName:}" failed. No retries permitted until 2026-07-10 00:26:53.831357071 +0000 UTC m=+34.274086120 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret") pod "global-pull-secret-syncer-6zh4c" (UID: "7886b79d-8489-44af-9072-36341cd95e39") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831386 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831405 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fl7vv for pod openshift-network-diagnostics/network-check-target-6k8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831405 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:26:53.831396012 +0000 UTC m=+34.274125054 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:37.831551 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:37.831474 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv podName:ddd92337-9f34-40a2-8c24-9049b923c61e nodeName:}" failed. No retries permitted until 2026-07-10 00:26:53.831450341 +0000 UTC m=+34.274179382 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fl7vv" (UniqueName: "kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv") pod "network-check-target-6k8f8" (UID: "ddd92337-9f34-40a2-8c24-9049b923c61e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:38.021475 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:38.021441 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:38.021656 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:38.021441 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:38.021656 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:38.021567 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:38.022083 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:38.021657 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:39.021442 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:39.021400 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:39.021620 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:39.021509 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:40.022048 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:40.021811 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:40.022496 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:40.021887 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:40.022496 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:40.022179 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:40.022496 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:40.022288 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:41.021545 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:41.021515 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:41.021702 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:41.021639 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:42.021449 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:42.021410 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:42.021449 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:42.021456 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:42.021987 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:42.021550 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:42.021987 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:42.021711 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:43.021396 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:43.021361 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:43.021625 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:43.021487 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:44.021104 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:44.021066 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:44.021309 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:44.021134 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:44.021309 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:44.021223 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:44.021433 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:44.021324 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:45.021316 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:45.021278 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:45.021782 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:45.021419 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:46.021052 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:46.021017 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:46.021241 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:46.021155 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:46.021241 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:46.021213 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:46.021385 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:46.021360 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:47.020781 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.020657 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:47.020899 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:47.020876 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:47.132782 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.132753 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" event={"ID":"e7299d47-d182-431c-b83d-8a4aa223f2de","Type":"ContainerStarted","Data":"fa03f065ecad6a829c8448926ccc669140a921d9b072c5b33db5dd1397a49512"} Jul 10 00:26:47.134588 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.134564 2579 generic.go:356] "Generic (PLEG): container finished" podID="12ba7873-dbe9-43a9-9f47-86f4280640c6" containerID="1bf0d91582ed02028ddb2a72ac043b05a6f7a350f46fc332cdfc68ad5eef8a8b" exitCode=0 Jul 10 00:26:47.134801 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.134643 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerDied","Data":"1bf0d91582ed02028ddb2a72ac043b05a6f7a350f46fc332cdfc68ad5eef8a8b"} Jul 10 00:26:47.139109 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.139091 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:26:47.139469 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.139453 2579 generic.go:356] "Generic (PLEG): container finished" podID="bbde56ff-c2db-4d64-886b-ff6d2554e16c" containerID="46056049a33a6299ff4d30812a3e02970776fd16e64614538bdce1a24d586afd" exitCode=1 Jul 10 00:26:47.139539 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.139515 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"622579276adf23c533aece3e12eaa94dd0b1decda4c2c8cf5cf5cd60d5971bc8"} Jul 10 00:26:47.139627 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.139612 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"56067b7650880191aea3bfa215895328d3f5207769fe5f200f2b47b22dee4541"} Jul 10 00:26:47.139681 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.139657 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"47fa17c849269b8d9bd78af02d92bc4d8163fbb060c113036843e18be302c1dc"} Jul 10 00:26:47.139782 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.139701 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerDied","Data":"46056049a33a6299ff4d30812a3e02970776fd16e64614538bdce1a24d586afd"} Jul 10 00:26:47.139782 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.139739 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"81e4e5e2fb635e1142d60ae20c4da8a5763fe352a52840363b4ff3bb06e5c981"} Jul 10 00:26:47.141167 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.141146 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m8ldd" event={"ID":"853e3939-f02a-467a-844d-b22ecfd84736","Type":"ContainerStarted","Data":"97fab5f8e3a296f2c530262fe4b0f126b02af08ee6605a7e566e91f15f29d85b"} Jul 10 00:26:47.150028 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.149982 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zm6mh" podStartSLOduration=2.332519152 podStartE2EDuration="26.149966893s" totalImagesPullingTime="23.817447741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:47.149757999 +0000 UTC m=+27.592487049" watchObservedRunningTime="2026-07-10 00:26:47.149966893 +0000 UTC m=+27.592695948" Jul 10 00:26:47.164655 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:47.164611 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-m8ldd" podStartSLOduration=2.748913914 podStartE2EDuration="27.164594944s" totalImagesPullingTime="24.41568103s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:47.164274333 +0000 UTC m=+27.607003377" watchObservedRunningTime="2026-07-10 00:26:47.164594944 +0000 UTC m=+27.607323992" Jul 10 00:26:48.021306 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:48.021227 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:48.021306 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:48.021268 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:48.021530 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:48.021368 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:48.021530 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:48.021444 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:48.148883 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:48.148851 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:26:48.149444 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:48.149217 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"7577c76ac8c05030da53b325b9e7d190504e03254583df6972d28770534fa7a1"} Jul 10 00:26:49.020532 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:49.020499 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:49.020757 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:49.020605 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:49.152957 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:49.152922 2579 generic.go:356] "Generic (PLEG): container finished" podID="12ba7873-dbe9-43a9-9f47-86f4280640c6" containerID="d034917fe07212b253119c2c5cb3105c2ef5e6a33037762b38a96c44834a435b" exitCode=0 Jul 10 00:26:49.153489 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:49.152962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerDied","Data":"d034917fe07212b253119c2c5cb3105c2ef5e6a33037762b38a96c44834a435b"} Jul 10 00:26:50.021614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:50.021571 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:50.021800 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:50.021677 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:50.021800 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:50.021724 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:50.021923 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:50.021825 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:50.158185 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:50.158161 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:26:50.158577 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:50.158470 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"c0e57c9f8aa1be3df97d539a9b5a23ae74cb5a11e743b067450f9f8f7066afb5"} Jul 10 00:26:51.021485 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:51.021451 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:51.021659 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:51.021573 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:51.162581 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:51.162544 2579 generic.go:356] "Generic (PLEG): container finished" podID="12ba7873-dbe9-43a9-9f47-86f4280640c6" containerID="c63ebcb28921a63851135b2bd4ad127fd5f29f223e7a902b642f0cf8efb0a156" exitCode=0 Jul 10 00:26:51.163149 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:51.162591 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerDied","Data":"c63ebcb28921a63851135b2bd4ad127fd5f29f223e7a902b642f0cf8efb0a156"} Jul 10 00:26:52.021605 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.021428 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:52.021774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.021428 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:52.021774 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:52.021724 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:52.021774 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:52.021755 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:52.169614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.169584 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:26:52.170030 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.169996 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"cc534032ab306957e6dd1b0e219297d88ab67b9a573a5ea216e46b9cce40cf37"} Jul 10 00:26:52.170321 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.170303 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:52.170417 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.170379 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:52.170483 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.170431 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:52.170533 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.170516 2579 scope.go:117] "RemoveContainer" containerID="46056049a33a6299ff4d30812a3e02970776fd16e64614538bdce1a24d586afd" Jul 10 00:26:52.191120 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.191092 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:52.191574 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:52.191553 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:26:53.020973 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.020942 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:53.021163 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.021057 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:53.176935 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.176909 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:26:53.177438 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.177343 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" event={"ID":"bbde56ff-c2db-4d64-886b-ff6d2554e16c","Type":"ContainerStarted","Data":"63322e3bc1410542c65dffd0d17db9bb6ef5c2b31f319c75a1960ce71f18f3aa"} Jul 10 00:26:53.622114 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.622051 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" podStartSLOduration=8.215612325 podStartE2EDuration="32.622027859s" totalImagesPullingTime="24.406415534s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:53.207726051 +0000 UTC m=+33.650455096" watchObservedRunningTime="2026-07-10 00:26:53.622027859 +0000 UTC m=+34.064756911" Jul 10 00:26:53.622563 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.622542 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6zh4c"] Jul 10 00:26:53.622694 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.622669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:53.622799 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.622779 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:53.625164 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.625140 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mqw4x"] Jul 10 00:26:53.625284 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.625200 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6k8f8"] Jul 10 00:26:53.625284 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.625276 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:53.625399 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.625366 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:53.625462 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.625442 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:53.625549 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.625531 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:53.850434 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.850403 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:53.850614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.850466 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:53.850614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:53.850493 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:53.850614 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850589 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 00:26:53.850614 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850614 2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 00:26:53.850781 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850621 2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:53.850781 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850629 2579 projected.go:196] Error preparing data for projected volume kube-api-access-fl7vv for pod openshift-network-diagnostics/network-check-target-6k8f8: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:53.850781 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850641 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:53.850781 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850685 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv podName:ddd92337-9f34-40a2-8c24-9049b923c61e nodeName:}" failed. No retries permitted until 2026-07-10 00:27:25.850668051 +0000 UTC m=+66.293397083 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fl7vv" (UniqueName: "kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv") pod "network-check-target-6k8f8" (UID: "ddd92337-9f34-40a2-8c24-9049b923c61e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 00:26:53.850781 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850700 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:27:25.850692785 +0000 UTC m=+66.293421815 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 00:26:53.850781 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:53.850711 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret podName:7886b79d-8489-44af-9072-36341cd95e39 nodeName:}" failed. No retries permitted until 2026-07-10 00:27:25.850705342 +0000 UTC m=+66.293434371 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret") pod "global-pull-secret-syncer-6zh4c" (UID: "7886b79d-8489-44af-9072-36341cd95e39") : object "kube-system"/"original-pull-secret" not registered Jul 10 00:26:55.021282 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:55.021229 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:55.021772 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:55.021289 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:55.021772 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:55.021383 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:55.021772 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:55.021504 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:56.021707 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:56.021669 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:56.022070 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:56.021809 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:57.021698 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:57.021482 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:57.021839 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:57.021551 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:57.021839 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:57.021802 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:57.022207 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:57.021863 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:57.189102 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:57.189065 2579 generic.go:356] "Generic (PLEG): container finished" podID="12ba7873-dbe9-43a9-9f47-86f4280640c6" containerID="ef0fd8a75f33296058ffe28a40004b5af6f96b4d514feb2a03ed8dd5ac1f8855" exitCode=0 Jul 10 00:26:57.189102 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:57.189109 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerDied","Data":"ef0fd8a75f33296058ffe28a40004b5af6f96b4d514feb2a03ed8dd5ac1f8855"} Jul 10 00:26:58.021207 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:58.021104 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:26:58.021384 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:58.021242 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6zh4c" podUID="7886b79d-8489-44af-9072-36341cd95e39" Jul 10 00:26:58.194151 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:58.194119 2579 generic.go:356] "Generic (PLEG): container finished" podID="12ba7873-dbe9-43a9-9f47-86f4280640c6" containerID="ea919085196949e7fdd9b37cb6456ad0dfbb6dea2faf5b360ce12217984cecfe" exitCode=0 Jul 10 00:26:58.194522 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:58.194165 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerDied","Data":"ea919085196949e7fdd9b37cb6456ad0dfbb6dea2faf5b360ce12217984cecfe"} Jul 10 00:26:59.020933 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.020896 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:26:59.021094 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.020912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:26:59.021094 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:59.020994 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6k8f8" podUID="ddd92337-9f34-40a2-8c24-9049b923c61e" Jul 10 00:26:59.021094 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:59.021084 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:26:59.199240 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.199205 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" event={"ID":"12ba7873-dbe9-43a9-9f47-86f4280640c6","Type":"ContainerStarted","Data":"56f6d0a111a3df0872efe0e7e92ebff51842a06fe6787c78ff294877acb3e1c4"} Jul 10 00:26:59.220269 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.219965 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qfwkw" podStartSLOduration=7.745431324 podStartE2EDuration="39.219946197s" totalImagesPullingTime="31.474514873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:26:59.218712697 +0000 UTC m=+39.661441748" watchObservedRunningTime="2026-07-10 00:26:59.219946197 +0000 UTC m=+39.662675249" Jul 10 00:26:59.393684 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.393659 2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-148.ec2.internal" event="NodeReady" Jul 10 00:26:59.393851 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.393772 2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 00:26:59.425437 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.425411 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd"] Jul 10 00:26:59.450616 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.450581 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67f798784-cbswt"] Jul 10 00:26:59.450767 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.450699 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.453344 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.453168 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 00:26:59.453344 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.453226 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 00:26:59.453344 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.453224 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 00:26:59.453344 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.453323 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 00:26:59.453603 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.453403 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-b26md\"" Jul 10 00:26:59.466632 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.466602 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt"] Jul 10 00:26:59.466759 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.466720 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.469188 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.469165 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 00:26:59.469337 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.469220 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 00:26:59.469401 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.469334 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 00:26:59.469401 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.469334 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6nz9c\"" Jul 10 00:26:59.477435 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.477415 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 00:26:59.484685 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.484667 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz"] Jul 10 00:26:59.484911 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.484890 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.487928 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.487902 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 10 00:26:59.510988 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.510959 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd"] Jul 10 00:26:59.511098 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.510998 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pm99w"] Jul 10 00:26:59.511098 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.511010 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.513179 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.513155 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 10 00:26:59.513295 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.513229 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 10 00:26:59.513295 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.513286 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 10 00:26:59.513390 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.513361 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 10 00:26:59.535036 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.535016 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lwpfk"] Jul 10 00:26:59.535151 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.535137 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:26:59.537375 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.537360 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 00:26:59.537602 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.537586 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 00:26:59.537684 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.537618 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5rckt\"" Jul 10 00:26:59.537684 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.537624 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 00:26:59.550986 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.550972 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz"] Jul 10 00:26:59.551038 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.550997 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f798784-cbswt"] Jul 10 00:26:59.551038 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.551017 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pm99w"] Jul 10 00:26:59.551038 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.551035 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt"] Jul 10 00:26:59.551150 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.551055 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lwpfk"] Jul 10 00:26:59.551150 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.551111 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.553413 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.553397 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 00:26:59.553502 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.553489 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 00:26:59.553559 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.553504 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2cgcq\"" Jul 10 00:26:59.594476 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594448 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-hub\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.594476 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594477 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksrp5\" (UniqueName: \"kubernetes.io/projected/a823c515-513a-419f-9b6c-18c090e2d7f5-kube-api-access-ksrp5\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.594641 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594493 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e375968a-2aa9-45d0-aa02-2149451f7675-tmp\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.594641 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594510 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bcpj\" (UniqueName: \"kubernetes.io/projected/e375968a-2aa9-45d0-aa02-2149451f7675-kube-api-access-4bcpj\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.594641 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594569 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2fe09e4-64f5-432e-ba7d-e4046b6b325b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74f5f5c698-nqgzd\" (UID: \"b2fe09e4-64f5-432e-ba7d-e4046b6b325b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.594641 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594588 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-ca\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.594778 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594638 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xghbh\" (UniqueName: \"kubernetes.io/projected/b2fe09e4-64f5-432e-ba7d-e4046b6b325b-kube-api-access-xghbh\") pod \"managed-serviceaccount-addon-agent-74f5f5c698-nqgzd\" (UID: \"b2fe09e4-64f5-432e-ba7d-e4046b6b325b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.594778 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594667 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8e16e0-0007-48a5-bd34-e311b2c871cc-ca-trust-extracted\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.594778 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594685 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-certificates\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.594778 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594703 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chbjs\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-kube-api-access-chbjs\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.594778 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594718 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e375968a-2aa9-45d0-aa02-2149451f7675-klusterlet-config\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.594778 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594751 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-installation-pull-secrets\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.594981 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594804 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-image-registry-private-configuration\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.594981 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594853 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.594981 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594873 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-trusted-ca\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.594981 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594894 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.594981 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594926 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a823c515-513a-419f-9b6c-18c090e2d7f5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.594981 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594942 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.595172 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.594980 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-bound-sa-token\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.695526 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695444 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-bound-sa-token\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.695526 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695482 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-hub\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.695526 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksrp5\" (UniqueName: \"kubernetes.io/projected/a823c515-513a-419f-9b6c-18c090e2d7f5-kube-api-access-ksrp5\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.695752 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695530 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e375968a-2aa9-45d0-aa02-2149451f7675-tmp\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.695752 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695694 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bcpj\" (UniqueName: \"kubernetes.io/projected/e375968a-2aa9-45d0-aa02-2149451f7675-kube-api-access-4bcpj\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.695752 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695730 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-config-volume\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.695881 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695761 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2fe09e4-64f5-432e-ba7d-e4046b6b325b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74f5f5c698-nqgzd\" (UID: \"b2fe09e4-64f5-432e-ba7d-e4046b6b325b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.695881 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695793 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-ca\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.695881 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xghbh\" (UniqueName: \"kubernetes.io/projected/b2fe09e4-64f5-432e-ba7d-e4046b6b325b-kube-api-access-xghbh\") pod \"managed-serviceaccount-addon-agent-74f5f5c698-nqgzd\" (UID: \"b2fe09e4-64f5-432e-ba7d-e4046b6b325b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.695881 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695854 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8e16e0-0007-48a5-bd34-e311b2c871cc-ca-trust-extracted\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696069 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695969 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-certificates\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696069 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.695996 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e375968a-2aa9-45d0-aa02-2149451f7675-tmp\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.696069 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696000 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59844dc5-d391-4c33-8851-e79dd39ffb86-cert\") pod \"ingress-canary-pm99w\" (UID: \"59844dc5-d391-4c33-8851-e79dd39ffb86\") " pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:26:59.696214 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696126 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdgbm\" (UniqueName: \"kubernetes.io/projected/59844dc5-d391-4c33-8851-e79dd39ffb86-kube-api-access-qdgbm\") pod \"ingress-canary-pm99w\" (UID: \"59844dc5-d391-4c33-8851-e79dd39ffb86\") " pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:26:59.696214 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696158 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chbjs\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-kube-api-access-chbjs\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696214 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696187 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e375968a-2aa9-45d0-aa02-2149451f7675-klusterlet-config\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.696214 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696209 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8e16e0-0007-48a5-bd34-e311b2c871cc-ca-trust-extracted\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696438 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696241 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-installation-pull-secrets\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696438 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696297 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv26j\" (UniqueName: \"kubernetes.io/projected/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-kube-api-access-xv26j\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.696438 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696360 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-image-registry-private-configuration\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696438 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696391 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696438 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696432 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-trusted-ca\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696631 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696460 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.696631 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696517 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a823c515-513a-419f-9b6c-18c090e2d7f5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.696631 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696544 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.696631 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696555 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-certificates\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.696631 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696592 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.696631 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.696616 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-tmp-dir\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.698194 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:59.697668 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:26:59.698194 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:59.697696 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:26:59.698194 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.697698 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-trusted-ca\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.698194 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:59.697762 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:27:00.197741893 +0000 UTC m=+40.640470925 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:26:59.698194 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.698168 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a823c515-513a-419f-9b6c-18c090e2d7f5-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.701023 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.700992 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-image-registry-private-configuration\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.701187 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.701045 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-installation-pull-secrets\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.701187 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.701072 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-hub\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.701187 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.701140 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.701446 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.701200 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.701446 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.701213 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2fe09e4-64f5-432e-ba7d-e4046b6b325b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74f5f5c698-nqgzd\" (UID: \"b2fe09e4-64f5-432e-ba7d-e4046b6b325b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.701547 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.701518 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a823c515-513a-419f-9b6c-18c090e2d7f5-ca\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.701609 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.701588 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e375968a-2aa9-45d0-aa02-2149451f7675-klusterlet-config\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.703950 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.703927 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bcpj\" (UniqueName: \"kubernetes.io/projected/e375968a-2aa9-45d0-aa02-2149451f7675-kube-api-access-4bcpj\") pod \"klusterlet-addon-workmgr-74cb4cd8db-22qtt\" (UID: \"e375968a-2aa9-45d0-aa02-2149451f7675\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.704296 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.704275 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-bound-sa-token\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.704433 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.704414 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksrp5\" (UniqueName: \"kubernetes.io/projected/a823c515-513a-419f-9b6c-18c090e2d7f5-kube-api-access-ksrp5\") pod \"cluster-proxy-proxy-agent-7966f54dbc-bm6lz\" (UID: \"a823c515-513a-419f-9b6c-18c090e2d7f5\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.704489 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.704424 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xghbh\" (UniqueName: \"kubernetes.io/projected/b2fe09e4-64f5-432e-ba7d-e4046b6b325b-kube-api-access-xghbh\") pod \"managed-serviceaccount-addon-agent-74f5f5c698-nqgzd\" (UID: \"b2fe09e4-64f5-432e-ba7d-e4046b6b325b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.704532 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.704485 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chbjs\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-kube-api-access-chbjs\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:26:59.768389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.768352 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:26:59.793669 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.793642 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:26:59.797894 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.797877 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-config-volume\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.797973 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.797925 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59844dc5-d391-4c33-8851-e79dd39ffb86-cert\") pod \"ingress-canary-pm99w\" (UID: \"59844dc5-d391-4c33-8851-e79dd39ffb86\") " pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:26:59.797973 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.797943 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdgbm\" (UniqueName: \"kubernetes.io/projected/59844dc5-d391-4c33-8851-e79dd39ffb86-kube-api-access-qdgbm\") pod \"ingress-canary-pm99w\" (UID: \"59844dc5-d391-4c33-8851-e79dd39ffb86\") " pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:26:59.798099 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.798080 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv26j\" (UniqueName: \"kubernetes.io/projected/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-kube-api-access-xv26j\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.798156 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.798134 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.798202 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.798163 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-tmp-dir\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.798300 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:59.798281 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:26:59.798361 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:26:59.798356 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:27:00.298338584 +0000 UTC m=+40.741067616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:26:59.798454 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.798437 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-tmp-dir\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.798543 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.798525 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-config-volume\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.800378 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.800362 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59844dc5-d391-4c33-8851-e79dd39ffb86-cert\") pod \"ingress-canary-pm99w\" (UID: \"59844dc5-d391-4c33-8851-e79dd39ffb86\") " pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:26:59.806452 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.806271 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdgbm\" (UniqueName: \"kubernetes.io/projected/59844dc5-d391-4c33-8851-e79dd39ffb86-kube-api-access-qdgbm\") pod \"ingress-canary-pm99w\" (UID: \"59844dc5-d391-4c33-8851-e79dd39ffb86\") " pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:26:59.806726 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.806708 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv26j\" (UniqueName: \"kubernetes.io/projected/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-kube-api-access-xv26j\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:26:59.818711 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.818687 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:26:59.861982 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:26:59.861954 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pm99w" Jul 10 00:27:00.024313 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.022714 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:27:00.027110 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.026909 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 00:27:00.027535 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.027514 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd"] Jul 10 00:27:00.035849 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.035824 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz"] Jul 10 00:27:00.038004 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.037986 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt"] Jul 10 00:27:00.045856 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:27:00.045825 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode375968a_2aa9_45d0_aa02_2149451f7675.slice/crio-9020ef281face52b62d6efb400ad1b3547cbeb0b88c28be12738d2eff6b66307 WatchSource:0}: Error finding container 9020ef281face52b62d6efb400ad1b3547cbeb0b88c28be12738d2eff6b66307: Status 404 returned error can't find the container with id 9020ef281face52b62d6efb400ad1b3547cbeb0b88c28be12738d2eff6b66307 Jul 10 00:27:00.072376 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.072349 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pm99w"] Jul 10 00:27:00.075570 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:27:00.075548 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59844dc5_d391_4c33_8851_e79dd39ffb86.slice/crio-dd0ab9fcb67ec79ae6296c95b6f64caac13f448997f3e934bd6bcc6eefae71c2 WatchSource:0}: Error finding container dd0ab9fcb67ec79ae6296c95b6f64caac13f448997f3e934bd6bcc6eefae71c2: Status 404 returned error can't find the container with id dd0ab9fcb67ec79ae6296c95b6f64caac13f448997f3e934bd6bcc6eefae71c2 Jul 10 00:27:00.202477 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.202390 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:27:00.203084 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:00.202546 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:27:00.203084 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:00.202560 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:27:00.203084 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:00.202610 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:27:01.20259604 +0000 UTC m=+41.645325069 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:27:00.203287 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.203267 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" event={"ID":"a823c515-513a-419f-9b6c-18c090e2d7f5","Type":"ContainerStarted","Data":"0979d5f9cfc591df5cf9c5312354c08453b6b76c9d97fc1d8c281b5800d82a97"} Jul 10 00:27:00.204303 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.204280 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" event={"ID":"b2fe09e4-64f5-432e-ba7d-e4046b6b325b","Type":"ContainerStarted","Data":"6c1d4086a35b9de4c99b1f73e20816c860602599f0d51cc101a089570e47f4b1"} Jul 10 00:27:00.205299 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.205280 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pm99w" event={"ID":"59844dc5-d391-4c33-8851-e79dd39ffb86","Type":"ContainerStarted","Data":"dd0ab9fcb67ec79ae6296c95b6f64caac13f448997f3e934bd6bcc6eefae71c2"} Jul 10 00:27:00.206290 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.206273 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" event={"ID":"e375968a-2aa9-45d0-aa02-2149451f7675","Type":"ContainerStarted","Data":"9020ef281face52b62d6efb400ad1b3547cbeb0b88c28be12738d2eff6b66307"} Jul 10 00:27:00.303329 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:00.303298 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:27:00.303506 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:00.303483 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:27:00.303570 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:00.303546 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:27:01.303527771 +0000 UTC m=+41.746256814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:27:01.022276 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.021570 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:27:01.022276 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.021600 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:27:01.026883 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.026852 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 00:27:01.027196 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.027174 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zkg6s\"" Jul 10 00:27:01.027308 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.027223 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 00:27:01.027530 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.027513 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 00:27:01.027602 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.027584 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5zcx\"" Jul 10 00:27:01.210715 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.210684 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:27:01.211404 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:01.211384 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:27:01.211404 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:01.211407 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:27:01.211546 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:01.211465 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:27:03.211446317 +0000 UTC m=+43.654175348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:27:01.311842 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:01.311758 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:27:01.312018 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:01.311993 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:27:01.312072 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:01.312059 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:27:03.312039967 +0000 UTC m=+43.754768998 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:27:03.232573 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:03.232532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:27:03.233015 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:03.232770 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:27:03.233015 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:03.232793 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:27:03.233015 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:03.232864 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:27:07.232847688 +0000 UTC m=+47.675576717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:27:03.333708 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:03.333033 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:27:03.333708 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:03.333271 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:27:03.333708 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:03.333337 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:27:07.333318305 +0000 UTC m=+47.776047339 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:27:06.226840 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:06.226634 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" event={"ID":"a823c515-513a-419f-9b6c-18c090e2d7f5","Type":"ContainerStarted","Data":"6a047ac37cf1473e66d0a05351c2050741794ee0eade9a853974b1643476b1bf"} Jul 10 00:27:06.228868 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:06.228475 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" event={"ID":"b2fe09e4-64f5-432e-ba7d-e4046b6b325b","Type":"ContainerStarted","Data":"903c7a7f2f76754dabb37ea49828ba6b85766c10116b8457c2df828d091fe9d8"} Jul 10 00:27:06.230321 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:06.229865 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pm99w" event={"ID":"59844dc5-d391-4c33-8851-e79dd39ffb86","Type":"ContainerStarted","Data":"571bbffe75f78f52c5c7b3f451efe45391ac41be6b74a2f3fe7453d532f89d9e"} Jul 10 00:27:06.245243 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:06.245195 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" podStartSLOduration=18.228757305 podStartE2EDuration="24.245179861s" totalImagesPullingTime="6.016422556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:06.243872444 +0000 UTC m=+46.686601498" watchObservedRunningTime="2026-07-10 00:27:06.245179861 +0000 UTC m=+46.687908890" Jul 10 00:27:06.256851 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:06.256791 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pm99w" podStartSLOduration=1.2893810399999999 podStartE2EDuration="7.256772892s" totalImagesPullingTime="5.967391852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:06.256619227 +0000 UTC m=+46.699348291" watchObservedRunningTime="2026-07-10 00:27:06.256772892 +0000 UTC m=+46.699501944" Jul 10 00:27:07.234166 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:07.234132 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" event={"ID":"a823c515-513a-419f-9b6c-18c090e2d7f5","Type":"ContainerStarted","Data":"02ef2cf4a6ce5efbc2a27bc13f012b95b365c3fe886f1ac13a96faf82faba141"} Jul 10 00:27:07.234758 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:07.234179 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" event={"ID":"a823c515-513a-419f-9b6c-18c090e2d7f5","Type":"ContainerStarted","Data":"bc26531b3b7bec2d298fdfa1e6fe02f143d2cb4150e3a89c924c6cc167c1012c"} Jul 10 00:27:07.251514 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:07.251471 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" podStartSLOduration=19.229485507 podStartE2EDuration="25.251457152s" totalImagesPullingTime="6.021971645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:07.250411926 +0000 UTC m=+47.693140975" watchObservedRunningTime="2026-07-10 00:27:07.251457152 +0000 UTC m=+47.694186182" Jul 10 00:27:07.265524 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:07.265503 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:27:07.265654 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:07.265639 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:27:07.265701 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:07.265657 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:27:07.265739 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:07.265703 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:27:15.265691196 +0000 UTC m=+55.708420224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:27:07.366370 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:07.366336 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:27:07.366561 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:07.366485 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:27:07.366561 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:07.366556 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:27:15.366537146 +0000 UTC m=+55.809266175 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:27:10.244512 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:10.244472 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" event={"ID":"e375968a-2aa9-45d0-aa02-2149451f7675","Type":"ContainerStarted","Data":"0b9ed9c22111dd72a36ad3e8ab288583281b5a2952bbb2ec34c79fe4830e3090"} Jul 10 00:27:10.244883 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:10.244612 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:27:10.246135 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:10.246119 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:27:10.258464 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:10.258421 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" podStartSLOduration=19.089955709 podStartE2EDuration="28.258405304s" totalImagesPullingTime="9.168449595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:10.258381355 +0000 UTC m=+50.701110406" watchObservedRunningTime="2026-07-10 00:27:10.258405304 +0000 UTC m=+50.701134356" Jul 10 00:27:15.326006 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:15.325966 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:27:15.326806 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:15.326114 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:27:15.326806 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:15.326133 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:27:15.326806 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:15.326197 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:27:31.326181299 +0000 UTC m=+71.768910327 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:27:15.427078 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:15.427045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:27:15.427242 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:15.427206 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:27:15.427306 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:15.427282 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:27:31.427265583 +0000 UTC m=+71.869994611 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:27:24.191138 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:24.191109 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tjxpd" Jul 10 00:27:25.906338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.906299 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:27:25.906338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.906357 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:27:25.906875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.906405 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:27:25.908805 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.908787 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 00:27:25.908963 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.908945 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 00:27:25.909056 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.909040 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 00:27:25.916773 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:25.916756 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 00:27:25.916838 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:25.916814 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:28:29.916793913 +0000 UTC m=+130.359522941 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : secret "metrics-daemon-secret" not found Jul 10 00:27:25.918774 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.918761 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 00:27:25.920855 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.920839 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7886b79d-8489-44af-9072-36341cd95e39-original-pull-secret\") pod \"global-pull-secret-syncer-6zh4c\" (UID: \"7886b79d-8489-44af-9072-36341cd95e39\") " pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:27:25.929937 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.929917 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl7vv\" (UniqueName: \"kubernetes.io/projected/ddd92337-9f34-40a2-8c24-9049b923c61e-kube-api-access-fl7vv\") pod \"network-check-target-6k8f8\" (UID: \"ddd92337-9f34-40a2-8c24-9049b923c61e\") " pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:27:25.954159 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.954134 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zkg6s\"" Jul 10 00:27:25.963020 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:25.963002 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:27:26.093369 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:26.093343 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6k8f8"] Jul 10 00:27:26.096754 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:27:26.096729 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddd92337_9f34_40a2_8c24_9049b923c61e.slice/crio-c5ab7602ed97c475c18a4cafb9ed685f964aecfa6ae90ef5056b626f01e9d943 WatchSource:0}: Error finding container c5ab7602ed97c475c18a4cafb9ed685f964aecfa6ae90ef5056b626f01e9d943: Status 404 returned error can't find the container with id c5ab7602ed97c475c18a4cafb9ed685f964aecfa6ae90ef5056b626f01e9d943 Jul 10 00:27:26.135880 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:26.135859 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6zh4c" Jul 10 00:27:26.267660 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:26.267627 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6zh4c"] Jul 10 00:27:26.282514 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:27:26.282488 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7886b79d_8489_44af_9072_36341cd95e39.slice/crio-c043cfa87b4cbe2387ed0dc29c1b27a5a6b15de1da3b3328d7d74bc68d81233b WatchSource:0}: Error finding container c043cfa87b4cbe2387ed0dc29c1b27a5a6b15de1da3b3328d7d74bc68d81233b: Status 404 returned error can't find the container with id c043cfa87b4cbe2387ed0dc29c1b27a5a6b15de1da3b3328d7d74bc68d81233b Jul 10 00:27:26.288915 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:26.288890 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6zh4c" event={"ID":"7886b79d-8489-44af-9072-36341cd95e39","Type":"ContainerStarted","Data":"c043cfa87b4cbe2387ed0dc29c1b27a5a6b15de1da3b3328d7d74bc68d81233b"} Jul 10 00:27:26.289989 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:26.289962 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6k8f8" event={"ID":"ddd92337-9f34-40a2-8c24-9049b923c61e","Type":"ContainerStarted","Data":"c5ab7602ed97c475c18a4cafb9ed685f964aecfa6ae90ef5056b626f01e9d943"} Jul 10 00:27:30.302667 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:30.302580 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6k8f8" event={"ID":"ddd92337-9f34-40a2-8c24-9049b923c61e","Type":"ContainerStarted","Data":"435c87ba984a5fabaae46608c2cd3dc08f61f6cf24594c5c209668894f4a1285"} Jul 10 00:27:30.303094 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:30.302702 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:27:30.318430 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:30.318379 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6k8f8" podStartSLOduration=66.374052945 podStartE2EDuration="1m10.31836005s" totalImagesPullingTime="3.944307105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:30.317440076 +0000 UTC m=+70.760169129" watchObservedRunningTime="2026-07-10 00:27:30.31836005 +0000 UTC m=+70.761089101" Jul 10 00:27:31.348875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:31.348838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:27:31.349447 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:31.349012 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:27:31.349447 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:31.349038 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:27:31.349447 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:31.349105 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:28:03.349087375 +0000 UTC m=+103.791816404 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:27:31.450347 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:31.450309 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:27:31.450516 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:31.450467 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:27:31.450576 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:27:31.450546 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:28:03.450525445 +0000 UTC m=+103.893254478 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:27:32.310212 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:32.310173 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6zh4c" event={"ID":"7886b79d-8489-44af-9072-36341cd95e39","Type":"ContainerStarted","Data":"5fbeda7238b4fbbb0da56ef09e1320505737c44a59fad941d442895c65fbbabd"} Jul 10 00:27:32.326744 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:27:32.326693 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6zh4c" podStartSLOduration=66.060573677 podStartE2EDuration="1m11.326678737s" totalImagesPullingTime="5.26610506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:27:32.325397902 +0000 UTC m=+72.768126952" watchObservedRunningTime="2026-07-10 00:27:32.326678737 +0000 UTC m=+72.769407830" Jul 10 00:28:01.308567 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:28:01.308448 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6k8f8" Jul 10 00:28:03.380696 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:28:03.380661 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:28:03.381231 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:28:03.380864 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:28:03.381231 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:28:03.380885 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:28:03.381231 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:28:03.380958 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:29:07.3809358 +0000 UTC m=+167.823664833 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:28:03.481567 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:28:03.481477 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:28:03.481710 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:28:03.481598 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:28:03.481710 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:28:03.481650 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:29:07.481635949 +0000 UTC m=+167.924364978 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:28:29.970377 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:28:29.970346 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:28:29.970824 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:28:29.970469 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 00:28:29.970824 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:28:29.970537 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:30:31.970520082 +0000 UTC m=+252.413249128 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : secret "metrics-daemon-secret" not found Jul 10 00:29:02.477561 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:02.477513 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-67f798784-cbswt" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" Jul 10 00:29:02.567675 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:02.567641 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lwpfk" podUID="ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8" Jul 10 00:29:02.569789 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:02.569770 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:29:03.572993 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:03.572971 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lwpfk" Jul 10 00:29:04.041228 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:04.041149 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:29:06.584298 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:06.584240 2579 generic.go:356] "Generic (PLEG): container finished" podID="b2fe09e4-64f5-432e-ba7d-e4046b6b325b" containerID="903c7a7f2f76754dabb37ea49828ba6b85766c10116b8457c2df828d091fe9d8" exitCode=255 Jul 10 00:29:06.584298 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:06.584298 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" event={"ID":"b2fe09e4-64f5-432e-ba7d-e4046b6b325b","Type":"ContainerDied","Data":"903c7a7f2f76754dabb37ea49828ba6b85766c10116b8457c2df828d091fe9d8"} Jul 10 00:29:06.584778 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:06.584566 2579 scope.go:117] "RemoveContainer" containerID="903c7a7f2f76754dabb37ea49828ba6b85766c10116b8457c2df828d091fe9d8" Jul 10 00:29:07.426494 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:07.426455 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:29:07.426672 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:07.426632 2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 00:29:07.426672 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:07.426653 2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67f798784-cbswt: secret "image-registry-tls" not found Jul 10 00:29:07.426791 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:07.426724 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls podName:ef8e16e0-0007-48a5-bd34-e311b2c871cc nodeName:}" failed. No retries permitted until 2026-07-10 00:31:09.426707219 +0000 UTC m=+289.869436249 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls") pod "image-registry-67f798784-cbswt" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc") : secret "image-registry-tls" not found Jul 10 00:29:07.527191 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:07.527136 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:29:07.527378 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:07.527281 2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 00:29:07.527378 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:29:07.527334 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls podName:ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8 nodeName:}" failed. No retries permitted until 2026-07-10 00:31:09.527319936 +0000 UTC m=+289.970048964 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls") pod "dns-default-lwpfk" (UID: "ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8") : secret "dns-default-metrics-tls" not found Jul 10 00:29:07.588366 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:07.588333 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" event={"ID":"b2fe09e4-64f5-432e-ba7d-e4046b6b325b","Type":"ContainerStarted","Data":"b1bcbe64fefd84c3dba58853711fb8517ed2fe1a10d3d1b243001f9b28c61d6d"} Jul 10 00:29:09.595864 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:09.595783 2579 generic.go:356] "Generic (PLEG): container finished" podID="e375968a-2aa9-45d0-aa02-2149451f7675" containerID="0b9ed9c22111dd72a36ad3e8ab288583281b5a2952bbb2ec34c79fe4830e3090" exitCode=1 Jul 10 00:29:09.595864 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:09.595823 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" event={"ID":"e375968a-2aa9-45d0-aa02-2149451f7675","Type":"ContainerDied","Data":"0b9ed9c22111dd72a36ad3e8ab288583281b5a2952bbb2ec34c79fe4830e3090"} Jul 10 00:29:09.596381 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:09.596100 2579 scope.go:117] "RemoveContainer" containerID="0b9ed9c22111dd72a36ad3e8ab288583281b5a2952bbb2ec34c79fe4830e3090" Jul 10 00:29:09.794197 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:09.794169 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:29:10.244982 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:10.244945 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:29:10.600938 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:10.600905 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" event={"ID":"e375968a-2aa9-45d0-aa02-2149451f7675","Type":"ContainerStarted","Data":"e36230daaf18fc276b2a59670b63bc5b5d98bdab0c078a50c037b72eddbaa9ad"} Jul 10 00:29:10.601362 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:10.601169 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:29:10.602536 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:10.602517 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:29:15.020815 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:29:15.020775 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:30:25.496428 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.496394 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kvzq5"] Jul 10 00:30:25.499372 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.499352 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.501536 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.501518 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 00:30:25.501728 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.501710 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 00:30:25.501882 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.501861 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g2hvb\"" Jul 10 00:30:25.501988 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.501973 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 00:30:25.502054 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.502002 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 00:30:25.511456 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.511418 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kvzq5"] Jul 10 00:30:25.574655 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.574626 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5a2d808-e0e7-475a-b3d6-a04be12797e5-crio-socket\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.574875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.574664 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jc9c\" (UniqueName: \"kubernetes.io/projected/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-api-access-2jc9c\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.574875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.574688 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5a2d808-e0e7-475a-b3d6-a04be12797e5-data-volume\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.574875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.574777 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.574875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.574814 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.675733 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.675696 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5a2d808-e0e7-475a-b3d6-a04be12797e5-crio-socket\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.675910 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.675748 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jc9c\" (UniqueName: \"kubernetes.io/projected/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-api-access-2jc9c\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.675910 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.675782 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5a2d808-e0e7-475a-b3d6-a04be12797e5-data-volume\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.675910 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.675820 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.675910 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.675822 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5a2d808-e0e7-475a-b3d6-a04be12797e5-crio-socket\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.675910 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.675851 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.675910 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:25.675899 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:30:25.676168 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:25.675973 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:30:25.676168 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:25.675987 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:26.175970568 +0000 UTC m=+246.618699597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : configmap "kube-rbac-proxy" not found Jul 10 00:30:25.676168 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:25.676032 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:26.176015746 +0000 UTC m=+246.618744779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:30:25.676168 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.676083 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5a2d808-e0e7-475a-b3d6-a04be12797e5-data-volume\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:25.684535 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:25.684506 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jc9c\" (UniqueName: \"kubernetes.io/projected/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-api-access-2jc9c\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:26.179615 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:26.179579 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:26.179763 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:26.179626 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:26.179763 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:26.179705 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:30:26.179855 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:26.179776 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:30:26.179855 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:26.179788 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:27.17976882 +0000 UTC m=+247.622497850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : configmap "kube-rbac-proxy" not found Jul 10 00:30:26.179855 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:26.179810 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:27.179800777 +0000 UTC m=+247.622529808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:30:27.187723 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:27.187686 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:27.188085 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:27.187731 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:27.188085 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:27.187810 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:30:27.188085 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:27.187861 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:30:27.188085 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:27.187877 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:29.187861927 +0000 UTC m=+249.630590957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : configmap "kube-rbac-proxy" not found Jul 10 00:30:27.188085 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:27.187904 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:29.187893853 +0000 UTC m=+249.630622883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:30:27.689552 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:27.689529 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xtbwc_07492d99-1fb7-4079-ac5a-1c9e41d1ed5c/dns-node-resolver/0.log" Jul 10 00:30:28.688883 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:28.688858 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmzlh_655a9d60-5c3a-455b-832e-56e8a1062561/node-ca/0.log" Jul 10 00:30:29.203886 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:29.203835 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:29.204097 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:29.203900 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:29.204097 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:29.203961 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:30:29.204097 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:29.204027 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:33.204011951 +0000 UTC m=+253.646740980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : configmap "kube-rbac-proxy" not found Jul 10 00:30:29.204097 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:29.204049 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:30:29.204097 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:29.204093 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:33.204081741 +0000 UTC m=+253.646810771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:30:29.488813 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:29.488748 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pm99w_59844dc5-d391-4c33-8851-e79dd39ffb86/serve-healthcheck-canary/0.log" Jul 10 00:30:32.025779 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:32.025746 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:30:32.026231 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:32.025864 2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 10 00:30:32.026231 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:32.025938 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs podName:d5f71bf8-0d90-4b94-84c2-e706db351d5a nodeName:}" failed. No retries permitted until 2026-07-10 00:32:34.025919584 +0000 UTC m=+374.468648629 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs") pod "network-metrics-daemon-mqw4x" (UID: "d5f71bf8-0d90-4b94-84c2-e706db351d5a") : secret "metrics-daemon-secret" not found Jul 10 00:30:33.234632 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:33.234591 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:33.235072 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:33.234642 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:33.235072 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:33.234705 2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 00:30:33.235072 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:33.234769 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:41.234754959 +0000 UTC m=+261.677483987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : configmap "kube-rbac-proxy" not found Jul 10 00:30:33.235072 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:33.234806 2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 00:30:33.235072 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:30:33.234854 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls podName:c5a2d808-e0e7-475a-b3d6-a04be12797e5 nodeName:}" failed. No retries permitted until 2026-07-10 00:30:41.234839311 +0000 UTC m=+261.677568340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvzq5" (UID: "c5a2d808-e0e7-475a-b3d6-a04be12797e5") : secret "insights-runtime-extractor-tls" not found Jul 10 00:30:41.294998 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:41.294915 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:41.295482 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:41.295008 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:41.295577 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:41.295557 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5a2d808-e0e7-475a-b3d6-a04be12797e5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:41.297381 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:41.297359 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5a2d808-e0e7-475a-b3d6-a04be12797e5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvzq5\" (UID: \"c5a2d808-e0e7-475a-b3d6-a04be12797e5\") " pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:41.408196 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:41.408164 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kvzq5" Jul 10 00:30:41.531520 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:41.531482 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kvzq5"] Jul 10 00:30:41.535298 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:30:41.535278 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5a2d808_e0e7_475a_b3d6_a04be12797e5.slice/crio-f9d58976c4bb1b73915c2db1bf78beb26bbd80f64fa54ef1e63e2bcfa6107308 WatchSource:0}: Error finding container f9d58976c4bb1b73915c2db1bf78beb26bbd80f64fa54ef1e63e2bcfa6107308: Status 404 returned error can't find the container with id f9d58976c4bb1b73915c2db1bf78beb26bbd80f64fa54ef1e63e2bcfa6107308 Jul 10 00:30:41.864473 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:41.864437 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvzq5" event={"ID":"c5a2d808-e0e7-475a-b3d6-a04be12797e5","Type":"ContainerStarted","Data":"f9d58976c4bb1b73915c2db1bf78beb26bbd80f64fa54ef1e63e2bcfa6107308"} Jul 10 00:30:45.879715 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:45.879692 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvzq5" event={"ID":"c5a2d808-e0e7-475a-b3d6-a04be12797e5","Type":"ContainerStarted","Data":"bf8f45d91c6745501f724bad71e28a456a45fd3ed4825aa749a5a9f58f80ebb5"} Jul 10 00:30:46.885221 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:46.885181 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvzq5" event={"ID":"c5a2d808-e0e7-475a-b3d6-a04be12797e5","Type":"ContainerStarted","Data":"30a0ced86b60b1efaf7da029fe79bac518fd50c8de8ccbc83e1e77f84eb333b7"} Jul 10 00:30:48.893926 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:48.893888 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvzq5" event={"ID":"c5a2d808-e0e7-475a-b3d6-a04be12797e5","Type":"ContainerStarted","Data":"99bf090ecfdfdc4d43d806d1d1aafe14c6905dcec80d3d680875f3c3f2db4555"} Jul 10 00:30:48.910411 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:30:48.910362 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kvzq5" podStartSLOduration=17.292203513 podStartE2EDuration="23.910346543s" totalImagesPullingTime="6.61814303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 00:30:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:30:48.908748251 +0000 UTC m=+269.351477300" watchObservedRunningTime="2026-07-10 00:30:48.910346543 +0000 UTC m=+269.353075592" Jul 10 00:31:01.966950 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.966921 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8nfhh"] Jul 10 00:31:01.970313 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.970290 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:01.972764 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.972736 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 00:31:01.972892 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.972794 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 00:31:01.972892 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.972832 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 00:31:01.972892 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.972848 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 00:31:01.973101 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.973086 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 00:31:01.973172 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.973087 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qzs5v\"" Jul 10 00:31:01.973559 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:01.973544 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 00:31:02.054663 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054635 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-root\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.054663 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054662 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-textfile\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.054865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054680 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-sys\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.054865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054694 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-tls\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.054865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054763 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvclw\" (UniqueName: \"kubernetes.io/projected/16a526ed-bb88-4551-9779-1d89ffd9cfaf-kube-api-access-lvclw\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.054865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054810 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.054865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054837 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.054865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054853 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a526ed-bb88-4551-9779-1d89ffd9cfaf-metrics-client-ca\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.055098 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.054946 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-wtmp\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156082 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156048 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156082 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156082 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a526ed-bb88-4551-9779-1d89ffd9cfaf-metrics-client-ca\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156112 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-wtmp\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156132 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-root\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156183 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-root\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156263 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-textfile\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156275 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-wtmp\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156304 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-sys\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156338 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156329 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-tls\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156654 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156376 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvclw\" (UniqueName: \"kubernetes.io/projected/16a526ed-bb88-4551-9779-1d89ffd9cfaf-kube-api-access-lvclw\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156654 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156388 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/16a526ed-bb88-4551-9779-1d89ffd9cfaf-sys\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156654 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156434 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156654 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:02.156488 2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 10 00:31:02.156654 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156550 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-textfile\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156654 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:02.156559 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-tls podName:16a526ed-bb88-4551-9779-1d89ffd9cfaf nodeName:}" failed. No retries permitted until 2026-07-10 00:31:02.656527571 +0000 UTC m=+283.099256605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-tls") pod "node-exporter-8nfhh" (UID: "16a526ed-bb88-4551-9779-1d89ffd9cfaf") : secret "node-exporter-tls" not found Jul 10 00:31:02.156856 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156714 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-accelerators-collector-config\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.156856 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.156732 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16a526ed-bb88-4551-9779-1d89ffd9cfaf-metrics-client-ca\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.158619 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.158604 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.166595 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.166573 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvclw\" (UniqueName: \"kubernetes.io/projected/16a526ed-bb88-4551-9779-1d89ffd9cfaf-kube-api-access-lvclw\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.660401 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.660369 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-tls\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.662692 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.662674 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/16a526ed-bb88-4551-9779-1d89ffd9cfaf-node-exporter-tls\") pod \"node-exporter-8nfhh\" (UID: \"16a526ed-bb88-4551-9779-1d89ffd9cfaf\") " pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.879593 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.879556 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8nfhh" Jul 10 00:31:02.892201 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:31:02.892174 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a526ed_bb88_4551_9779_1d89ffd9cfaf.slice/crio-35604b3a4bdc1443d52f83444f0588df68edf079261116822a6b337f7c8db5df WatchSource:0}: Error finding container 35604b3a4bdc1443d52f83444f0588df68edf079261116822a6b337f7c8db5df: Status 404 returned error can't find the container with id 35604b3a4bdc1443d52f83444f0588df68edf079261116822a6b337f7c8db5df Jul 10 00:31:02.943354 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:02.943272 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8nfhh" event={"ID":"16a526ed-bb88-4551-9779-1d89ffd9cfaf","Type":"ContainerStarted","Data":"35604b3a4bdc1443d52f83444f0588df68edf079261116822a6b337f7c8db5df"} Jul 10 00:31:04.950502 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:04.950470 2579 generic.go:356] "Generic (PLEG): container finished" podID="16a526ed-bb88-4551-9779-1d89ffd9cfaf" containerID="77d6bd66af9e7142f5863f9b5a062bbb4aecebf5a20a1341889b185cf0fd5e16" exitCode=0 Jul 10 00:31:04.950855 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:04.950519 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8nfhh" event={"ID":"16a526ed-bb88-4551-9779-1d89ffd9cfaf","Type":"ContainerDied","Data":"77d6bd66af9e7142f5863f9b5a062bbb4aecebf5a20a1341889b185cf0fd5e16"} Jul 10 00:31:05.570597 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:05.570543 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-67f798784-cbswt" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" Jul 10 00:31:05.955410 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:05.955373 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8nfhh" event={"ID":"16a526ed-bb88-4551-9779-1d89ffd9cfaf","Type":"ContainerStarted","Data":"d279725fe2c1d8c432376c4673805129b9c7813b3cf2da18f7bf3045cacb11e7"} Jul 10 00:31:05.955790 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:05.955449 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:31:05.955790 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:05.955474 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8nfhh" event={"ID":"16a526ed-bb88-4551-9779-1d89ffd9cfaf","Type":"ContainerStarted","Data":"522c8b0efb3456f84e1840678149b7a5b5a0d8582854f0258d838bea113d108b"} Jul 10 00:31:05.972835 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:05.972677 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8nfhh" podStartSLOduration=3.882875838 podStartE2EDuration="4.97265784s" totalImagesPullingTime="1.020542425s" totalInitContainerRuntime="69.239577ms" isStatefulPod=true podCreationTimestamp="2026-07-10 00:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:31:05.971500137 +0000 UTC m=+286.414229187" watchObservedRunningTime="2026-07-10 00:31:05.97265784 +0000 UTC m=+286.415386892" Jul 10 00:31:06.573824 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:06.573776 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-lwpfk" podUID="ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8" Jul 10 00:31:06.960398 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:06.960300 2579 generic.go:356] "Generic (PLEG): container finished" podID="b2fe09e4-64f5-432e-ba7d-e4046b6b325b" containerID="b1bcbe64fefd84c3dba58853711fb8517ed2fe1a10d3d1b243001f9b28c61d6d" exitCode=255 Jul 10 00:31:06.960398 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:06.960380 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" event={"ID":"b2fe09e4-64f5-432e-ba7d-e4046b6b325b","Type":"ContainerDied","Data":"b1bcbe64fefd84c3dba58853711fb8517ed2fe1a10d3d1b243001f9b28c61d6d"} Jul 10 00:31:06.960882 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:06.960435 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lwpfk" Jul 10 00:31:06.960882 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:06.960441 2579 scope.go:117] "RemoveContainer" containerID="903c7a7f2f76754dabb37ea49828ba6b85766c10116b8457c2df828d091fe9d8" Jul 10 00:31:06.960959 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:06.960892 2579 scope.go:117] "RemoveContainer" containerID="b1bcbe64fefd84c3dba58853711fb8517ed2fe1a10d3d1b243001f9b28c61d6d" Jul 10 00:31:06.961053 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:06.961031 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-74f5f5c698-nqgzd_open-cluster-management-agent-addon(b2fe09e4-64f5-432e-ba7d-e4046b6b325b)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" podUID="b2fe09e4-64f5-432e-ba7d-e4046b6b325b" Jul 10 00:31:09.513080 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.513034 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:31:09.515477 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.515454 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"image-registry-67f798784-cbswt\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:31:09.557478 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.557455 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6nz9c\"" Jul 10 00:31:09.566599 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.566583 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:31:09.613608 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.613583 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:31:09.616190 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.616161 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8-metrics-tls\") pod \"dns-default-lwpfk\" (UID: \"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8\") " pod="openshift-dns/dns-default-lwpfk" Jul 10 00:31:09.663658 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.663633 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2cgcq\"" Jul 10 00:31:09.672485 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.672462 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lwpfk" Jul 10 00:31:09.693533 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.693496 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67f798784-cbswt"] Jul 10 00:31:09.751726 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:31:09.751693 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8e16e0_0007_48a5_bd34_e311b2c871cc.slice/crio-2c61df53add6c6346e776a07f0b871ac9a429402755662b5fc0798a74433c756 WatchSource:0}: Error finding container 2c61df53add6c6346e776a07f0b871ac9a429402755662b5fc0798a74433c756: Status 404 returned error can't find the container with id 2c61df53add6c6346e776a07f0b871ac9a429402755662b5fc0798a74433c756 Jul 10 00:31:09.769115 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.769075 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" Jul 10 00:31:09.769583 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.769567 2579 scope.go:117] "RemoveContainer" containerID="b1bcbe64fefd84c3dba58853711fb8517ed2fe1a10d3d1b243001f9b28c61d6d" Jul 10 00:31:09.769777 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:09.769736 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-74f5f5c698-nqgzd_open-cluster-management-agent-addon(b2fe09e4-64f5-432e-ba7d-e4046b6b325b)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" podUID="b2fe09e4-64f5-432e-ba7d-e4046b6b325b" Jul 10 00:31:09.794930 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.794901 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" podUID="e375968a-2aa9-45d0-aa02-2149451f7675" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/healthz\": dial tcp 10.132.0.9:8000: connect: connection refused" Jul 10 00:31:09.796974 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.796956 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lwpfk"] Jul 10 00:31:09.799690 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:31:09.799667 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce31d46b_cb81_4d6c_8b9d_d6f4fbff58a8.slice/crio-5f76bf81cb0668b0beb441ca230673bccdd5efcdbfcd009b25cf1dc8ae8e3bc1 WatchSource:0}: Error finding container 5f76bf81cb0668b0beb441ca230673bccdd5efcdbfcd009b25cf1dc8ae8e3bc1: Status 404 returned error can't find the container with id 5f76bf81cb0668b0beb441ca230673bccdd5efcdbfcd009b25cf1dc8ae8e3bc1 Jul 10 00:31:09.819871 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.819845 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" podUID="a823c515-513a-419f-9b6c-18c090e2d7f5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 10 00:31:09.973342 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.973306 2579 generic.go:356] "Generic (PLEG): container finished" podID="e375968a-2aa9-45d0-aa02-2149451f7675" containerID="e36230daaf18fc276b2a59670b63bc5b5d98bdab0c078a50c037b72eddbaa9ad" exitCode=1 Jul 10 00:31:09.973488 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.973383 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" event={"ID":"e375968a-2aa9-45d0-aa02-2149451f7675","Type":"ContainerDied","Data":"e36230daaf18fc276b2a59670b63bc5b5d98bdab0c078a50c037b72eddbaa9ad"} Jul 10 00:31:09.973488 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.973472 2579 scope.go:117] "RemoveContainer" containerID="0b9ed9c22111dd72a36ad3e8ab288583281b5a2952bbb2ec34c79fe4830e3090" Jul 10 00:31:09.973823 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.973766 2579 scope.go:117] "RemoveContainer" containerID="e36230daaf18fc276b2a59670b63bc5b5d98bdab0c078a50c037b72eddbaa9ad" Jul 10 00:31:09.973987 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:09.973964 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-74cb4cd8db-22qtt_open-cluster-management-agent-addon(e375968a-2aa9-45d0-aa02-2149451f7675)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" podUID="e375968a-2aa9-45d0-aa02-2149451f7675" Jul 10 00:31:09.974984 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.974955 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f798784-cbswt" event={"ID":"ef8e16e0-0007-48a5-bd34-e311b2c871cc","Type":"ContainerStarted","Data":"aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054"} Jul 10 00:31:09.975084 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.974999 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f798784-cbswt" event={"ID":"ef8e16e0-0007-48a5-bd34-e311b2c871cc","Type":"ContainerStarted","Data":"2c61df53add6c6346e776a07f0b871ac9a429402755662b5fc0798a74433c756"} Jul 10 00:31:09.975084 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.975023 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:31:09.976003 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:09.975987 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lwpfk" event={"ID":"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8","Type":"ContainerStarted","Data":"5f76bf81cb0668b0beb441ca230673bccdd5efcdbfcd009b25cf1dc8ae8e3bc1"} Jul 10 00:31:10.040817 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:10.040707 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67f798784-cbswt" podStartSLOduration=289.040686762 podStartE2EDuration="4m49.040686762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:31:10.039401543 +0000 UTC m=+290.482130581" watchObservedRunningTime="2026-07-10 00:31:10.040686762 +0000 UTC m=+290.483415812" Jul 10 00:31:10.601425 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:10.601394 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:31:10.980707 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:10.980628 2579 scope.go:117] "RemoveContainer" containerID="e36230daaf18fc276b2a59670b63bc5b5d98bdab0c078a50c037b72eddbaa9ad" Jul 10 00:31:10.980864 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:10.980814 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-74cb4cd8db-22qtt_open-cluster-management-agent-addon(e375968a-2aa9-45d0-aa02-2149451f7675)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" podUID="e375968a-2aa9-45d0-aa02-2149451f7675" Jul 10 00:31:11.988401 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:11.988298 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lwpfk" event={"ID":"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8","Type":"ContainerStarted","Data":"a15fd998375653bfc93eb0f7951f8c7767f794df9b14d735ef132d85e7e8f462"} Jul 10 00:31:12.994009 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:12.993977 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lwpfk" event={"ID":"ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8","Type":"ContainerStarted","Data":"f69729c54d4624df86841e8c4c33f72d5554513923e1d0eebfabf02cdc1bb831"} Jul 10 00:31:12.994489 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:12.994105 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lwpfk" Jul 10 00:31:13.010645 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:13.010600 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-lwpfk" podStartSLOduration=251.9471294 podStartE2EDuration="4m14.010584586s" totalImagesPullingTime="2.063455186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:31:13.009592725 +0000 UTC m=+293.452321775" watchObservedRunningTime="2026-07-10 00:31:13.010584586 +0000 UTC m=+293.453313637" Jul 10 00:31:18.021804 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:31:18.021764 2579 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mqw4x" podUID="d5f71bf8-0d90-4b94-84c2-e706db351d5a" Jul 10 00:31:19.793723 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:19.793685 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:31:19.794092 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:19.794005 2579 scope.go:117] "RemoveContainer" containerID="e36230daaf18fc276b2a59670b63bc5b5d98bdab0c078a50c037b72eddbaa9ad" Jul 10 00:31:19.820320 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:19.820288 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" podUID="a823c515-513a-419f-9b6c-18c090e2d7f5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 10 00:31:19.899047 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:19.899020 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:31:19.899553 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:19.899532 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:31:19.902794 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:19.902777 2579 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 00:31:21.019891 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:21.019858 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" event={"ID":"e375968a-2aa9-45d0-aa02-2149451f7675","Type":"ContainerStarted","Data":"42bed229d5ae0e2b1050c9d0b3e72fbcda59d09ffb50bddfef5a9d5a4eaa71a9"} Jul 10 00:31:21.020382 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:21.020159 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:31:21.021590 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:21.021571 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74cb4cd8db-22qtt" Jul 10 00:31:22.021237 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:22.021209 2579 scope.go:117] "RemoveContainer" containerID="b1bcbe64fefd84c3dba58853711fb8517ed2fe1a10d3d1b243001f9b28c61d6d" Jul 10 00:31:22.999943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:22.999912 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lwpfk" Jul 10 00:31:23.031124 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:23.031094 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74f5f5c698-nqgzd" event={"ID":"b2fe09e4-64f5-432e-ba7d-e4046b6b325b","Type":"ContainerStarted","Data":"e13570c91302a46663909ba24673cb7bdf9940af4c22b3af05910ae4d25b7c7d"} Jul 10 00:31:29.570851 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:29.570819 2579 patch_prober.go:28] interesting pod/image-registry-67f798784-cbswt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 10 00:31:29.571238 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:29.570879 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-67f798784-cbswt" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 00:31:29.820018 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:29.819978 2579 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" podUID="a823c515-513a-419f-9b6c-18c090e2d7f5" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 10 00:31:29.820173 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:29.820053 2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" Jul 10 00:31:29.820620 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:29.820591 2579 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"02ef2cf4a6ce5efbc2a27bc13f012b95b365c3fe886f1ac13a96faf82faba141"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 10 00:31:29.820679 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:29.820646 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" podUID="a823c515-513a-419f-9b6c-18c090e2d7f5" containerName="service-proxy" containerID="cri-o://02ef2cf4a6ce5efbc2a27bc13f012b95b365c3fe886f1ac13a96faf82faba141" gracePeriod=30 Jul 10 00:31:30.052942 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:30.052910 2579 generic.go:356] "Generic (PLEG): container finished" podID="a823c515-513a-419f-9b6c-18c090e2d7f5" containerID="02ef2cf4a6ce5efbc2a27bc13f012b95b365c3fe886f1ac13a96faf82faba141" exitCode=2 Jul 10 00:31:30.053103 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:30.052974 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" event={"ID":"a823c515-513a-419f-9b6c-18c090e2d7f5","Type":"ContainerDied","Data":"02ef2cf4a6ce5efbc2a27bc13f012b95b365c3fe886f1ac13a96faf82faba141"} Jul 10 00:31:30.053103 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:30.053012 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7966f54dbc-bm6lz" event={"ID":"a823c515-513a-419f-9b6c-18c090e2d7f5","Type":"ContainerStarted","Data":"6dae4e5369559a40edd336a558f7d7125471c7789cfa3f08123fc1b3a2c077b6"} Jul 10 00:31:30.984891 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:30.984865 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:31:31.021527 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:31.021491 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:31:36.422206 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:31:36.422174 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f798784-cbswt"] Jul 10 00:32:01.441775 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:01.441737 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67f798784-cbswt" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" containerName="registry" containerID="cri-o://aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054" gracePeriod=30 Jul 10 00:32:02.678782 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.678758 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:32:02.807939 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.807840 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-trusted-ca\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.807939 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.807888 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-bound-sa-token\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.807939 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.807913 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.807939 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.807941 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8e16e0-0007-48a5-bd34-e311b2c871cc-ca-trust-extracted\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.808285 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.807978 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-installation-pull-secrets\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.808285 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.808003 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-certificates\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.808285 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.808075 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-image-registry-private-configuration\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.808285 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.808102 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chbjs\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-kube-api-access-chbjs\") pod \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\" (UID: \"ef8e16e0-0007-48a5-bd34-e311b2c871cc\") " Jul 10 00:32:02.808501 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.808421 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:32:02.808562 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.808536 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:32:02.810823 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.810791 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-kube-api-access-chbjs" (OuterVolumeSpecName: "kube-api-access-chbjs") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "kube-api-access-chbjs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:32:02.810823 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.810799 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:32:02.810992 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.810840 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:32:02.810992 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.810857 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:32:02.810992 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.810973 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:32:02.817311 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.817285 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef8e16e0-0007-48a5-bd34-e311b2c871cc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ef8e16e0-0007-48a5-bd34-e311b2c871cc" (UID: "ef8e16e0-0007-48a5-bd34-e311b2c871cc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:32:02.909257 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909227 2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-image-registry-private-configuration\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:02.909460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909272 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chbjs\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-kube-api-access-chbjs\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:02.909460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909283 2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-trusted-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:02.909460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909291 2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-bound-sa-token\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:02.909460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909300 2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:02.909460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909308 2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ef8e16e0-0007-48a5-bd34-e311b2c871cc-ca-trust-extracted\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:02.909460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909316 2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ef8e16e0-0007-48a5-bd34-e311b2c871cc-installation-pull-secrets\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:02.909460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:02.909324 2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ef8e16e0-0007-48a5-bd34-e311b2c871cc-registry-certificates\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:32:03.157399 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.157368 2579 generic.go:356] "Generic (PLEG): container finished" podID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" containerID="aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054" exitCode=0 Jul 10 00:32:03.157548 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.157432 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67f798784-cbswt" Jul 10 00:32:03.157548 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.157444 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f798784-cbswt" event={"ID":"ef8e16e0-0007-48a5-bd34-e311b2c871cc","Type":"ContainerDied","Data":"aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054"} Jul 10 00:32:03.157548 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.157485 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67f798784-cbswt" event={"ID":"ef8e16e0-0007-48a5-bd34-e311b2c871cc","Type":"ContainerDied","Data":"2c61df53add6c6346e776a07f0b871ac9a429402755662b5fc0798a74433c756"} Jul 10 00:32:03.157548 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.157504 2579 scope.go:117] "RemoveContainer" containerID="aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054" Jul 10 00:32:03.166006 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.165987 2579 scope.go:117] "RemoveContainer" containerID="aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054" Jul 10 00:32:03.166306 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:32:03.166286 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054\": container with ID starting with aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054 not found: ID does not exist" containerID="aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054" Jul 10 00:32:03.166364 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.166315 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054"} err="failed to get container status \"aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054\": rpc error: code = NotFound desc = could not find container \"aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054\": container with ID starting with aa66528e56a035f554a5d5d3da8897851622165c56ba8d17e9b12afdd8769054 not found: ID does not exist" Jul 10 00:32:03.177354 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.177334 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67f798784-cbswt"] Jul 10 00:32:03.180294 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:03.180273 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67f798784-cbswt"] Jul 10 00:32:04.024859 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:04.024821 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" path="/var/lib/kubelet/pods/ef8e16e0-0007-48a5-bd34-e311b2c871cc/volumes" Jul 10 00:32:34.032934 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:34.032903 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:32:34.035282 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:34.035240 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d5f71bf8-0d90-4b94-84c2-e706db351d5a-metrics-certs\") pod \"network-metrics-daemon-mqw4x\" (UID: \"d5f71bf8-0d90-4b94-84c2-e706db351d5a\") " pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:32:34.324637 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:34.324614 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-d5zcx\"" Jul 10 00:32:34.332567 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:34.332538 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mqw4x" Jul 10 00:32:34.454938 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:34.454907 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mqw4x"] Jul 10 00:32:34.459194 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:32:34.459154 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f71bf8_0d90_4b94_84c2_e706db351d5a.slice/crio-0fc70a298d774ae1cc116c0fedcfbabcd7a20ae92303fec1e533b02a07596ae4 WatchSource:0}: Error finding container 0fc70a298d774ae1cc116c0fedcfbabcd7a20ae92303fec1e533b02a07596ae4: Status 404 returned error can't find the container with id 0fc70a298d774ae1cc116c0fedcfbabcd7a20ae92303fec1e533b02a07596ae4 Jul 10 00:32:34.460876 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:34.460859 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:32:35.257014 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:35.256981 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mqw4x" event={"ID":"d5f71bf8-0d90-4b94-84c2-e706db351d5a","Type":"ContainerStarted","Data":"0fc70a298d774ae1cc116c0fedcfbabcd7a20ae92303fec1e533b02a07596ae4"} Jul 10 00:32:37.265623 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:37.265582 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mqw4x" event={"ID":"d5f71bf8-0d90-4b94-84c2-e706db351d5a","Type":"ContainerStarted","Data":"4fc25ed793e51cb3460648c2cdbd8af9b6af3d56442133d5b26a5781df995bcf"} Jul 10 00:32:37.265623 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:32:37.265632 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mqw4x" event={"ID":"d5f71bf8-0d90-4b94-84c2-e706db351d5a","Type":"ContainerStarted","Data":"c8129ef2e5744aca1774ebfcaa9d1924d0e299588890443af8d99d2a52258386"} Jul 10 00:35:48.476345 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.476226 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mqw4x" podStartSLOduration=566.704785795 podStartE2EDuration="9m28.476205294s" totalImagesPullingTime="1.771419499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:32:37.28036588 +0000 UTC m=+377.723094934" watchObservedRunningTime="2026-07-10 00:35:48.476205294 +0000 UTC m=+568.918934345" Jul 10 00:35:48.477387 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.477365 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tj8zt"] Jul 10 00:35:48.477625 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.477610 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" containerName="registry" Jul 10 00:35:48.477691 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.477628 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" containerName="registry" Jul 10 00:35:48.477740 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.477701 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef8e16e0-0007-48a5-bd34-e311b2c871cc" containerName="registry" Jul 10 00:35:48.480194 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.480175 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.486487 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.486470 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 00:35:48.486597 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.486469 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 00:35:48.486597 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.486545 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hpnb9\"" Jul 10 00:35:48.501704 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.501679 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tj8zt"] Jul 10 00:35:48.536020 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.535996 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxpft\" (UniqueName: \"kubernetes.io/projected/331cedc6-12c6-465a-aa3a-e00de8fdfb18-kube-api-access-rxpft\") pod \"cert-manager-55b4d954d5-tj8zt\" (UID: \"331cedc6-12c6-465a-aa3a-e00de8fdfb18\") " pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.536173 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.536038 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/331cedc6-12c6-465a-aa3a-e00de8fdfb18-bound-sa-token\") pod \"cert-manager-55b4d954d5-tj8zt\" (UID: \"331cedc6-12c6-465a-aa3a-e00de8fdfb18\") " pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.637228 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.637185 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxpft\" (UniqueName: \"kubernetes.io/projected/331cedc6-12c6-465a-aa3a-e00de8fdfb18-kube-api-access-rxpft\") pod \"cert-manager-55b4d954d5-tj8zt\" (UID: \"331cedc6-12c6-465a-aa3a-e00de8fdfb18\") " pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.637420 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.637277 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/331cedc6-12c6-465a-aa3a-e00de8fdfb18-bound-sa-token\") pod \"cert-manager-55b4d954d5-tj8zt\" (UID: \"331cedc6-12c6-465a-aa3a-e00de8fdfb18\") " pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.646085 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.646052 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/331cedc6-12c6-465a-aa3a-e00de8fdfb18-bound-sa-token\") pod \"cert-manager-55b4d954d5-tj8zt\" (UID: \"331cedc6-12c6-465a-aa3a-e00de8fdfb18\") " pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.646206 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.646114 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxpft\" (UniqueName: \"kubernetes.io/projected/331cedc6-12c6-465a-aa3a-e00de8fdfb18-kube-api-access-rxpft\") pod \"cert-manager-55b4d954d5-tj8zt\" (UID: \"331cedc6-12c6-465a-aa3a-e00de8fdfb18\") " pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.788444 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.788354 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tj8zt" Jul 10 00:35:48.914771 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:48.914731 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tj8zt"] Jul 10 00:35:48.917625 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:35:48.917602 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod331cedc6_12c6_465a_aa3a_e00de8fdfb18.slice/crio-b213f25e1bc6e33a9dd3ae0793c5c608aa8064493f872454f88c035fc3466b68 WatchSource:0}: Error finding container b213f25e1bc6e33a9dd3ae0793c5c608aa8064493f872454f88c035fc3466b68: Status 404 returned error can't find the container with id b213f25e1bc6e33a9dd3ae0793c5c608aa8064493f872454f88c035fc3466b68 Jul 10 00:35:49.891747 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:49.891696 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tj8zt" event={"ID":"331cedc6-12c6-465a-aa3a-e00de8fdfb18","Type":"ContainerStarted","Data":"b213f25e1bc6e33a9dd3ae0793c5c608aa8064493f872454f88c035fc3466b68"} Jul 10 00:35:51.901208 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:51.901175 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tj8zt" event={"ID":"331cedc6-12c6-465a-aa3a-e00de8fdfb18","Type":"ContainerStarted","Data":"292792b841d14cf9d1a1f42796f26f6364fca0410294254f7c435840f8cf493e"} Jul 10 00:35:51.923036 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:35:51.922987 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-tj8zt" podStartSLOduration=1.013375108 podStartE2EDuration="3.922968879s" totalImagesPullingTime="2.909593771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:35:51.921370201 +0000 UTC m=+572.364099251" watchObservedRunningTime="2026-07-10 00:35:51.922968879 +0000 UTC m=+572.365697928" Jul 10 00:36:09.638569 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.638535 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs"] Jul 10 00:36:09.643618 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.643602 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.646114 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.646095 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 00:36:09.646460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.646440 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 00:36:09.646587 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.646572 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tlvtm\"" Jul 10 00:36:09.646794 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.646778 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 00:36:09.646875 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.646800 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 00:36:09.661259 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.661225 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs"] Jul 10 00:36:09.690062 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.690035 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-webhook-cert\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.690176 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.690067 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-tmp\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.690176 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.690086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-manifests\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.690176 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.690160 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlt9b\" (UniqueName: \"kubernetes.io/projected/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-kube-api-access-tlt9b\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.690290 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.690216 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.791484 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.791440 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-webhook-cert\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.791484 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.791489 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-tmp\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.791713 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.791525 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-manifests\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.791713 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.791552 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlt9b\" (UniqueName: \"kubernetes.io/projected/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-kube-api-access-tlt9b\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.791713 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.791575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.792044 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.792026 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-manifests\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.794007 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.793981 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-tmp\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.794153 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.794132 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.794211 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.794139 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-webhook-cert\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.806605 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.806575 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlt9b\" (UniqueName: \"kubernetes.io/projected/bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48-kube-api-access-tlt9b\") pod \"opendatahub-operator-controller-manager-54ccd4f945-dclxs\" (UID: \"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48\") " pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:09.953427 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:09.953341 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:10.091390 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:10.091351 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs"] Jul 10 00:36:10.096010 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:36:10.095985 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf8c998_b03d_4e9f_a3bf_b2bafc2fdb48.slice/crio-82dbdfe14bd70fb6b83907faab805caead1045606daf6346feb545cbd5689bba WatchSource:0}: Error finding container 82dbdfe14bd70fb6b83907faab805caead1045606daf6346feb545cbd5689bba: Status 404 returned error can't find the container with id 82dbdfe14bd70fb6b83907faab805caead1045606daf6346feb545cbd5689bba Jul 10 00:36:10.966056 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:10.965976 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" event={"ID":"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48","Type":"ContainerStarted","Data":"82dbdfe14bd70fb6b83907faab805caead1045606daf6346feb545cbd5689bba"} Jul 10 00:36:13.979913 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:13.979877 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" event={"ID":"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48","Type":"ContainerStarted","Data":"e774dd239cd30ec67ad325bcbee5b8b84f110875111612fa2eee8f748d67c42f"} Jul 10 00:36:14.984896 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:14.984860 2579 generic.go:356] "Generic (PLEG): container finished" podID="bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48" containerID="e774dd239cd30ec67ad325bcbee5b8b84f110875111612fa2eee8f748d67c42f" exitCode=0 Jul 10 00:36:14.985288 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:14.984919 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" event={"ID":"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48","Type":"ContainerDied","Data":"e774dd239cd30ec67ad325bcbee5b8b84f110875111612fa2eee8f748d67c42f"} Jul 10 00:36:15.990221 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:15.990186 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" event={"ID":"bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48","Type":"ContainerStarted","Data":"b5730e20c5a297f53997554e51d23f8c6945eb25503002c32ce6835c975f9af4"} Jul 10 00:36:15.990611 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:15.990323 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:16.009493 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:16.009450 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" podStartSLOduration=2.578702695 podStartE2EDuration="7.009434998s" totalImagesPullingTime="3.424058317s" totalInitContainerRuntime="1.006673986s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:36:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:36:16.008532889 +0000 UTC m=+596.451261942" watchObservedRunningTime="2026-07-10 00:36:16.009434998 +0000 UTC m=+596.452164046" Jul 10 00:36:19.927419 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:19.927390 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:36:19.928270 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:19.928229 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:36:26.996719 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:26.996680 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54ccd4f945-dclxs" Jul 10 00:36:27.689109 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.689076 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl"] Jul 10 00:36:27.692067 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.692052 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.696705 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.696680 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 10 00:36:27.696705 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.696706 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7tjhb\"" Jul 10 00:36:27.696892 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.696733 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 10 00:36:27.696892 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.696690 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 10 00:36:27.696892 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.696876 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 10 00:36:27.697036 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.696916 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 10 00:36:27.705192 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.705169 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl"] Jul 10 00:36:27.825029 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.824987 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90b13924-06cf-43a0-bd14-6b5acc4f212f-manager-config\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.825029 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.825031 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90b13924-06cf-43a0-bd14-6b5acc4f212f-cert\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.825285 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.825056 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvx6z\" (UniqueName: \"kubernetes.io/projected/90b13924-06cf-43a0-bd14-6b5acc4f212f-kube-api-access-hvx6z\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.825285 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.825112 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90b13924-06cf-43a0-bd14-6b5acc4f212f-metrics-cert\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.925621 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.925578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90b13924-06cf-43a0-bd14-6b5acc4f212f-manager-config\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.925621 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.925621 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90b13924-06cf-43a0-bd14-6b5acc4f212f-cert\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.925872 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.925639 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvx6z\" (UniqueName: \"kubernetes.io/projected/90b13924-06cf-43a0-bd14-6b5acc4f212f-kube-api-access-hvx6z\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.925872 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.925656 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90b13924-06cf-43a0-bd14-6b5acc4f212f-metrics-cert\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.926316 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.926293 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/90b13924-06cf-43a0-bd14-6b5acc4f212f-manager-config\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.928267 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.928224 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/90b13924-06cf-43a0-bd14-6b5acc4f212f-cert\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.928362 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.928272 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/90b13924-06cf-43a0-bd14-6b5acc4f212f-metrics-cert\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:27.934844 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:27.934825 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvx6z\" (UniqueName: \"kubernetes.io/projected/90b13924-06cf-43a0-bd14-6b5acc4f212f-kube-api-access-hvx6z\") pod \"lws-controller-manager-54495c74ff-5cstl\" (UID: \"90b13924-06cf-43a0-bd14-6b5acc4f212f\") " pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:28.000976 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:28.000877 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:28.128578 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:28.128545 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl"] Jul 10 00:36:28.131417 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:36:28.131386 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b13924_06cf_43a0_bd14_6b5acc4f212f.slice/crio-98ac690cc9969632a4172f7d17c49b800ed7032867abadc2191562b67dca3bc1 WatchSource:0}: Error finding container 98ac690cc9969632a4172f7d17c49b800ed7032867abadc2191562b67dca3bc1: Status 404 returned error can't find the container with id 98ac690cc9969632a4172f7d17c49b800ed7032867abadc2191562b67dca3bc1 Jul 10 00:36:29.037904 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:29.037869 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" event={"ID":"90b13924-06cf-43a0-bd14-6b5acc4f212f","Type":"ContainerStarted","Data":"98ac690cc9969632a4172f7d17c49b800ed7032867abadc2191562b67dca3bc1"} Jul 10 00:36:32.053792 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:32.053754 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" event={"ID":"90b13924-06cf-43a0-bd14-6b5acc4f212f","Type":"ContainerStarted","Data":"a9160166dd18bae30729487fa919028a0b18aff3fd29241527eca930812c5315"} Jul 10 00:36:32.054284 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:32.053902 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:36:32.069261 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:32.069197 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" podStartSLOduration=1.6902511489999998 podStartE2EDuration="5.069182152s" totalImagesPullingTime="3.378931003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:36:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:36:32.068241577 +0000 UTC m=+612.510970628" watchObservedRunningTime="2026-07-10 00:36:32.069182152 +0000 UTC m=+612.511911275" Jul 10 00:36:37.145102 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.145070 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw"] Jul 10 00:36:37.148152 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.148136 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.150453 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.150426 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 00:36:37.150575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.150529 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 00:36:37.150575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.150555 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 00:36:37.150575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.150570 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 00:36:37.150729 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.150528 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kzx28\"" Jul 10 00:36:37.150729 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.150625 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 00:36:37.150729 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.150650 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 00:36:37.161358 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.161339 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw"] Jul 10 00:36:37.294443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.294413 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.294443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.294451 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.294659 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.294482 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.294659 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.294530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.294659 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.294561 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.294659 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.294596 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k7hv\" (UniqueName: \"kubernetes.io/projected/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-kube-api-access-4k7hv\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.294775 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.294678 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.395267 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395165 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k7hv\" (UniqueName: \"kubernetes.io/projected/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-kube-api-access-4k7hv\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.395267 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.395443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395281 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.395443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395311 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.395443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395351 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.395443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395374 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.395443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395394 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.396011 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.395989 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.397610 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.397594 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.397872 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.397857 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.398216 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.398199 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.398344 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.398322 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.403343 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.403322 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k7hv\" (UniqueName: \"kubernetes.io/projected/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-kube-api-access-4k7hv\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.403638 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.403619 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5261ad2f-3b2c-4d15-a04a-d609b3555ad7-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gbdhw\" (UID: \"5261ad2f-3b2c-4d15-a04a-d609b3555ad7\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.457695 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.457664 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:37.588372 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:37.588339 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw"] Jul 10 00:36:37.592326 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:36:37.592293 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5261ad2f_3b2c_4d15_a04a_d609b3555ad7.slice/crio-70cd68efda6c8e6c337f87da6c6aafc289a9b2eb0071bea24788fab273bc3cb2 WatchSource:0}: Error finding container 70cd68efda6c8e6c337f87da6c6aafc289a9b2eb0071bea24788fab273bc3cb2: Status 404 returned error can't find the container with id 70cd68efda6c8e6c337f87da6c6aafc289a9b2eb0071bea24788fab273bc3cb2 Jul 10 00:36:38.074981 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:38.074951 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" event={"ID":"5261ad2f-3b2c-4d15-a04a-d609b3555ad7","Type":"ContainerStarted","Data":"70cd68efda6c8e6c337f87da6c6aafc289a9b2eb0071bea24788fab273bc3cb2"} Jul 10 00:36:41.180846 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:41.180810 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:36:41.181128 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:41.180875 2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 10 00:36:42.091797 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:42.091762 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" event={"ID":"5261ad2f-3b2c-4d15-a04a-d609b3555ad7","Type":"ContainerStarted","Data":"dcf1561b46e26ad209054f2375259f361239c13ba609b700c6f055cf3e98bc2a"} Jul 10 00:36:42.092015 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:42.091998 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:42.093701 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:42.093682 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" Jul 10 00:36:42.144466 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:42.144418 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gbdhw" podStartSLOduration=1.558136058 podStartE2EDuration="5.144401946s" totalImagesPullingTime="3.586265888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:36:42.140431923 +0000 UTC m=+622.583160973" watchObservedRunningTime="2026-07-10 00:36:42.144401946 +0000 UTC m=+622.587130988" Jul 10 00:36:43.059735 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:36:43.059707 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54495c74ff-5cstl" Jul 10 00:38:31.980998 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:31.980961 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4"] Jul 10 00:38:31.984144 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:31.984124 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:31.986736 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:31.986715 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 00:38:31.986857 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:31.986769 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 00:38:31.986857 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:31.986809 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-m8lwk\"" Jul 10 00:38:31.999718 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:31.999685 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4"] Jul 10 00:38:32.026844 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.026812 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f560fa-db2a-4a75-9598-2422fc807dcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-db2w4\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:32.027172 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.027138 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsk7v\" (UniqueName: \"kubernetes.io/projected/26f560fa-db2a-4a75-9598-2422fc807dcd-kube-api-access-dsk7v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-db2w4\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:32.127823 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.127783 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsk7v\" (UniqueName: \"kubernetes.io/projected/26f560fa-db2a-4a75-9598-2422fc807dcd-kube-api-access-dsk7v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-db2w4\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:32.127823 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.127827 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f560fa-db2a-4a75-9598-2422fc807dcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-db2w4\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:32.128157 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.128141 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f560fa-db2a-4a75-9598-2422fc807dcd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-db2w4\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:32.136662 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.136630 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsk7v\" (UniqueName: \"kubernetes.io/projected/26f560fa-db2a-4a75-9598-2422fc807dcd-kube-api-access-dsk7v\") pod \"kuadrant-operator-controller-manager-8684c9ff49-db2w4\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:32.294144 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.294052 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:32.428512 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.428480 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4"] Jul 10 00:38:32.431859 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:38:32.431826 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26f560fa_db2a_4a75_9598_2422fc807dcd.slice/crio-3081eeca8f27f0357367b62d9b90a9691a1ac65c46c155026e1f1d138ce8533d WatchSource:0}: Error finding container 3081eeca8f27f0357367b62d9b90a9691a1ac65c46c155026e1f1d138ce8533d: Status 404 returned error can't find the container with id 3081eeca8f27f0357367b62d9b90a9691a1ac65c46c155026e1f1d138ce8533d Jul 10 00:38:32.434161 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.434146 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:38:32.467468 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:32.467429 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" event={"ID":"26f560fa-db2a-4a75-9598-2422fc807dcd","Type":"ContainerStarted","Data":"3081eeca8f27f0357367b62d9b90a9691a1ac65c46c155026e1f1d138ce8533d"} Jul 10 00:38:38.495261 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:38.495221 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" event={"ID":"26f560fa-db2a-4a75-9598-2422fc807dcd","Type":"ContainerStarted","Data":"fb9309551225186cc637d2605a7ccd9ab7c28237357cde774f597141a4feddf7"} Jul 10 00:38:38.495709 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:38.495370 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:38.517055 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:38.516850 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" podStartSLOduration=2.326200199 podStartE2EDuration="7.51683349s" totalImagesPullingTime="5.190633291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:38:38.514882355 +0000 UTC m=+738.957611405" watchObservedRunningTime="2026-07-10 00:38:38.51683349 +0000 UTC m=+738.959562541" Jul 10 00:38:44.372410 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.372378 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4"] Jul 10 00:38:44.372817 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.372573 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" podUID="26f560fa-db2a-4a75-9598-2422fc807dcd" containerName="manager" containerID="cri-o://fb9309551225186cc637d2605a7ccd9ab7c28237357cde774f597141a4feddf7" gracePeriod=10 Jul 10 00:38:44.374267 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.374225 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:44.519387 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.519108 2579 generic.go:356] "Generic (PLEG): container finished" podID="26f560fa-db2a-4a75-9598-2422fc807dcd" containerID="fb9309551225186cc637d2605a7ccd9ab7c28237357cde774f597141a4feddf7" exitCode=0 Jul 10 00:38:44.519387 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.519206 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" event={"ID":"26f560fa-db2a-4a75-9598-2422fc807dcd","Type":"ContainerDied","Data":"fb9309551225186cc637d2605a7ccd9ab7c28237357cde774f597141a4feddf7"} Jul 10 00:38:44.621356 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.621329 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:44.733483 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.733387 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dsk7v\" (UniqueName: \"kubernetes.io/projected/26f560fa-db2a-4a75-9598-2422fc807dcd-kube-api-access-dsk7v\") pod \"26f560fa-db2a-4a75-9598-2422fc807dcd\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " Jul 10 00:38:44.733483 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.733461 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f560fa-db2a-4a75-9598-2422fc807dcd-extensions-socket-volume\") pod \"26f560fa-db2a-4a75-9598-2422fc807dcd\" (UID: \"26f560fa-db2a-4a75-9598-2422fc807dcd\") " Jul 10 00:38:44.733922 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.733885 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26f560fa-db2a-4a75-9598-2422fc807dcd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "26f560fa-db2a-4a75-9598-2422fc807dcd" (UID: "26f560fa-db2a-4a75-9598-2422fc807dcd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 00:38:44.735682 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.735658 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26f560fa-db2a-4a75-9598-2422fc807dcd-kube-api-access-dsk7v" (OuterVolumeSpecName: "kube-api-access-dsk7v") pod "26f560fa-db2a-4a75-9598-2422fc807dcd" (UID: "26f560fa-db2a-4a75-9598-2422fc807dcd"). InnerVolumeSpecName "kube-api-access-dsk7v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:38:44.834439 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.834404 2579 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/26f560fa-db2a-4a75-9598-2422fc807dcd-extensions-socket-volume\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:38:44.834439 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:44.834436 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dsk7v\" (UniqueName: \"kubernetes.io/projected/26f560fa-db2a-4a75-9598-2422fc807dcd-kube-api-access-dsk7v\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:38:45.524163 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:45.524125 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" event={"ID":"26f560fa-db2a-4a75-9598-2422fc807dcd","Type":"ContainerDied","Data":"3081eeca8f27f0357367b62d9b90a9691a1ac65c46c155026e1f1d138ce8533d"} Jul 10 00:38:45.524604 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:45.524138 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4" Jul 10 00:38:45.524604 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:45.524178 2579 scope.go:117] "RemoveContainer" containerID="fb9309551225186cc637d2605a7ccd9ab7c28237357cde774f597141a4feddf7" Jul 10 00:38:45.550157 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:45.550130 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4"] Jul 10 00:38:45.556793 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:45.556771 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-db2w4"] Jul 10 00:38:46.025187 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:38:46.025153 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26f560fa-db2a-4a75-9598-2422fc807dcd" path="/var/lib/kubelet/pods/26f560fa-db2a-4a75-9598-2422fc807dcd/volumes" Jul 10 00:39:13.539557 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.539518 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m7p59"] Jul 10 00:39:13.540006 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.539783 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26f560fa-db2a-4a75-9598-2422fc807dcd" containerName="manager" Jul 10 00:39:13.540006 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.539794 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="26f560fa-db2a-4a75-9598-2422fc807dcd" containerName="manager" Jul 10 00:39:13.540006 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.539839 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="26f560fa-db2a-4a75-9598-2422fc807dcd" containerName="manager" Jul 10 00:39:13.541620 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.541604 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.543857 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.543836 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 00:39:13.544614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.544597 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 00:39:13.544614 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.544607 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kmd5v\"" Jul 10 00:39:13.544729 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.544607 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 00:39:13.550030 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.550012 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m7p59"] Jul 10 00:39:13.554026 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.554006 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afd11a80-c514-48e4-b4f9-959a304a7423-config-file\") pod \"limitador-limitador-64c94fd64f-m7p59\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.554106 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.554081 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx85k\" (UniqueName: \"kubernetes.io/projected/afd11a80-c514-48e4-b4f9-959a304a7423-kube-api-access-lx85k\") pod \"limitador-limitador-64c94fd64f-m7p59\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.636581 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.636542 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m7p59"] Jul 10 00:39:13.654434 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.654412 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx85k\" (UniqueName: \"kubernetes.io/projected/afd11a80-c514-48e4-b4f9-959a304a7423-kube-api-access-lx85k\") pod \"limitador-limitador-64c94fd64f-m7p59\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.654554 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.654449 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afd11a80-c514-48e4-b4f9-959a304a7423-config-file\") pod \"limitador-limitador-64c94fd64f-m7p59\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.654997 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.654980 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afd11a80-c514-48e4-b4f9-959a304a7423-config-file\") pod \"limitador-limitador-64c94fd64f-m7p59\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.662279 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.662222 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx85k\" (UniqueName: \"kubernetes.io/projected/afd11a80-c514-48e4-b4f9-959a304a7423-kube-api-access-lx85k\") pod \"limitador-limitador-64c94fd64f-m7p59\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.851640 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.851606 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:13.974745 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:13.974716 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m7p59"] Jul 10 00:39:13.977848 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:39:13.977824 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafd11a80_c514_48e4_b4f9_959a304a7423.slice/crio-d29cc99959d414476a23420fa7e632751d27bac32c1e3a40644c89c9f9b37642 WatchSource:0}: Error finding container d29cc99959d414476a23420fa7e632751d27bac32c1e3a40644c89c9f9b37642: Status 404 returned error can't find the container with id d29cc99959d414476a23420fa7e632751d27bac32c1e3a40644c89c9f9b37642 Jul 10 00:39:14.634155 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:14.634122 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" event={"ID":"afd11a80-c514-48e4-b4f9-959a304a7423","Type":"ContainerStarted","Data":"d29cc99959d414476a23420fa7e632751d27bac32c1e3a40644c89c9f9b37642"} Jul 10 00:39:17.648437 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:17.648400 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" event={"ID":"afd11a80-c514-48e4-b4f9-959a304a7423","Type":"ContainerStarted","Data":"1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6"} Jul 10 00:39:17.648846 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:17.648520 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:17.664542 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:17.664493 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" podStartSLOduration=1.936085325 podStartE2EDuration="4.664478907s" totalImagesPullingTime="2.728393582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:39:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:39:17.663959243 +0000 UTC m=+778.106688295" watchObservedRunningTime="2026-07-10 00:39:17.664478907 +0000 UTC m=+778.107207958" Jul 10 00:39:28.654301 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:28.654271 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:28.818716 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:28.818683 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m7p59"] Jul 10 00:39:28.818908 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:28.818868 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" podUID="afd11a80-c514-48e4-b4f9-959a304a7423" containerName="limitador" containerID="cri-o://1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6" gracePeriod=30 Jul 10 00:39:29.360707 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.360686 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:29.470826 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.470738 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx85k\" (UniqueName: \"kubernetes.io/projected/afd11a80-c514-48e4-b4f9-959a304a7423-kube-api-access-lx85k\") pod \"afd11a80-c514-48e4-b4f9-959a304a7423\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " Jul 10 00:39:29.470826 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.470794 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afd11a80-c514-48e4-b4f9-959a304a7423-config-file\") pod \"afd11a80-c514-48e4-b4f9-959a304a7423\" (UID: \"afd11a80-c514-48e4-b4f9-959a304a7423\") " Jul 10 00:39:29.471171 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.471146 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afd11a80-c514-48e4-b4f9-959a304a7423-config-file" (OuterVolumeSpecName: "config-file") pod "afd11a80-c514-48e4-b4f9-959a304a7423" (UID: "afd11a80-c514-48e4-b4f9-959a304a7423"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:39:29.472998 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.472980 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afd11a80-c514-48e4-b4f9-959a304a7423-kube-api-access-lx85k" (OuterVolumeSpecName: "kube-api-access-lx85k") pod "afd11a80-c514-48e4-b4f9-959a304a7423" (UID: "afd11a80-c514-48e4-b4f9-959a304a7423"). InnerVolumeSpecName "kube-api-access-lx85k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:39:29.572035 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.571999 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx85k\" (UniqueName: \"kubernetes.io/projected/afd11a80-c514-48e4-b4f9-959a304a7423-kube-api-access-lx85k\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:39:29.572035 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.572028 2579 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/afd11a80-c514-48e4-b4f9-959a304a7423-config-file\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:39:29.689790 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.689757 2579 generic.go:356] "Generic (PLEG): container finished" podID="afd11a80-c514-48e4-b4f9-959a304a7423" containerID="1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6" exitCode=0 Jul 10 00:39:29.689790 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.689792 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" event={"ID":"afd11a80-c514-48e4-b4f9-959a304a7423","Type":"ContainerDied","Data":"1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6"} Jul 10 00:39:29.690327 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.689808 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" Jul 10 00:39:29.690327 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.689829 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m7p59" event={"ID":"afd11a80-c514-48e4-b4f9-959a304a7423","Type":"ContainerDied","Data":"d29cc99959d414476a23420fa7e632751d27bac32c1e3a40644c89c9f9b37642"} Jul 10 00:39:29.690327 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.689850 2579 scope.go:117] "RemoveContainer" containerID="1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6" Jul 10 00:39:29.698226 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.698207 2579 scope.go:117] "RemoveContainer" containerID="1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6" Jul 10 00:39:29.698479 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:39:29.698444 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6\": container with ID starting with 1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6 not found: ID does not exist" containerID="1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6" Jul 10 00:39:29.698547 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.698484 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6"} err="failed to get container status \"1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6\": rpc error: code = NotFound desc = could not find container \"1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6\": container with ID starting with 1663f2e581a5cdae311f83e8949bebd5a90feedf60d1967b293757efa4d917b6 not found: ID does not exist" Jul 10 00:39:29.710430 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.710408 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m7p59"] Jul 10 00:39:29.713764 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:29.713744 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m7p59"] Jul 10 00:39:30.024971 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:30.024939 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afd11a80-c514-48e4-b4f9-959a304a7423" path="/var/lib/kubelet/pods/afd11a80-c514-48e4-b4f9-959a304a7423/volumes" Jul 10 00:39:34.465721 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.465689 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z"] Jul 10 00:39:34.466071 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.466034 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afd11a80-c514-48e4-b4f9-959a304a7423" containerName="limitador" Jul 10 00:39:34.466071 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.466048 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="afd11a80-c514-48e4-b4f9-959a304a7423" containerName="limitador" Jul 10 00:39:34.466136 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.466122 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="afd11a80-c514-48e4-b4f9-959a304a7423" containerName="limitador" Jul 10 00:39:34.470563 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.470540 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.473147 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.473124 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 00:39:34.474051 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.474033 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-4hdlj\"" Jul 10 00:39:34.474291 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.474277 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 00:39:34.474373 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.474319 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 10 00:39:34.479956 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.479932 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z"] Jul 10 00:39:34.509085 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.509060 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mn5r\" (UniqueName: \"kubernetes.io/projected/41848e61-3f3d-4558-9e42-0d5ad1a880e8-kube-api-access-5mn5r\") pod \"postgres-69fb4594cf-ptk2z\" (UID: \"41848e61-3f3d-4558-9e42-0d5ad1a880e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.509178 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.509099 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/41848e61-3f3d-4558-9e42-0d5ad1a880e8-data\") pod \"postgres-69fb4594cf-ptk2z\" (UID: \"41848e61-3f3d-4558-9e42-0d5ad1a880e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.610081 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.610042 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mn5r\" (UniqueName: \"kubernetes.io/projected/41848e61-3f3d-4558-9e42-0d5ad1a880e8-kube-api-access-5mn5r\") pod \"postgres-69fb4594cf-ptk2z\" (UID: \"41848e61-3f3d-4558-9e42-0d5ad1a880e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.610287 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.610096 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/41848e61-3f3d-4558-9e42-0d5ad1a880e8-data\") pod \"postgres-69fb4594cf-ptk2z\" (UID: \"41848e61-3f3d-4558-9e42-0d5ad1a880e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.610476 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.610458 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/41848e61-3f3d-4558-9e42-0d5ad1a880e8-data\") pod \"postgres-69fb4594cf-ptk2z\" (UID: \"41848e61-3f3d-4558-9e42-0d5ad1a880e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.618756 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.618733 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mn5r\" (UniqueName: \"kubernetes.io/projected/41848e61-3f3d-4558-9e42-0d5ad1a880e8-kube-api-access-5mn5r\") pod \"postgres-69fb4594cf-ptk2z\" (UID: \"41848e61-3f3d-4558-9e42-0d5ad1a880e8\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.781191 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.781114 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:34.908682 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:34.908659 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z"] Jul 10 00:39:34.911164 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:39:34.911136 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41848e61_3f3d_4558_9e42_0d5ad1a880e8.slice/crio-59ed1e81efbf8eb0c02484831ab5107bd6d4cc5543c95a24afb2113ad9de6246 WatchSource:0}: Error finding container 59ed1e81efbf8eb0c02484831ab5107bd6d4cc5543c95a24afb2113ad9de6246: Status 404 returned error can't find the container with id 59ed1e81efbf8eb0c02484831ab5107bd6d4cc5543c95a24afb2113ad9de6246 Jul 10 00:39:35.715232 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:35.715154 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" event={"ID":"41848e61-3f3d-4558-9e42-0d5ad1a880e8","Type":"ContainerStarted","Data":"59ed1e81efbf8eb0c02484831ab5107bd6d4cc5543c95a24afb2113ad9de6246"} Jul 10 00:39:40.736071 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:40.735989 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" event={"ID":"41848e61-3f3d-4558-9e42-0d5ad1a880e8","Type":"ContainerStarted","Data":"b585b5a822e54d5e9fea958591fba54d7b2b88f11b4e8146688e4979cd7275f8"} Jul 10 00:39:40.736428 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:40.736085 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:39:40.752918 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:40.752875 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" podStartSLOduration=1.9640961670000001 podStartE2EDuration="6.752861682s" totalImagesPullingTime="4.788765515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:39:40.751185272 +0000 UTC m=+801.193914322" watchObservedRunningTime="2026-07-10 00:39:40.752861682 +0000 UTC m=+801.195590731" Jul 10 00:39:46.768540 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:39:46.768513 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ptk2z" Jul 10 00:40:33.763792 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.763761 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c5447fb99-fcplc"] Jul 10 00:40:33.769523 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.769494 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.772133 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.772114 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-486w4\"" Jul 10 00:40:33.772133 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.772128 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 10 00:40:33.775679 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.775659 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c5447fb99-fcplc"] Jul 10 00:40:33.857283 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.857238 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxfgk\" (UniqueName: \"kubernetes.io/projected/c5a020d6-f9a5-4e0a-8844-93373f810970-kube-api-access-vxfgk\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.857436 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.857305 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a020d6-f9a5-4e0a-8844-93373f810970-maas-api-tls\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.857436 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.857368 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a020d6-f9a5-4e0a-8844-93373f810970-openshift-service-ca\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.958010 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.957972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a020d6-f9a5-4e0a-8844-93373f810970-maas-api-tls\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.958010 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.958014 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a020d6-f9a5-4e0a-8844-93373f810970-openshift-service-ca\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.958206 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.958045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxfgk\" (UniqueName: \"kubernetes.io/projected/c5a020d6-f9a5-4e0a-8844-93373f810970-kube-api-access-vxfgk\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.958620 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.958597 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a020d6-f9a5-4e0a-8844-93373f810970-openshift-service-ca\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.960507 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.960487 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a020d6-f9a5-4e0a-8844-93373f810970-maas-api-tls\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:33.967068 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:33.967042 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxfgk\" (UniqueName: \"kubernetes.io/projected/c5a020d6-f9a5-4e0a-8844-93373f810970-kube-api-access-vxfgk\") pod \"maas-api-c5447fb99-fcplc\" (UID: \"c5a020d6-f9a5-4e0a-8844-93373f810970\") " pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:34.080506 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.080472 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:34.217028 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.216992 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c5447fb99-fcplc"] Jul 10 00:40:34.221093 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:40:34.221068 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5a020d6_f9a5_4e0a_8844_93373f810970.slice/crio-d4e65943d65a1bf4d073628766773372f3dfce9e09386a77810d6fbda137173a WatchSource:0}: Error finding container d4e65943d65a1bf4d073628766773372f3dfce9e09386a77810d6fbda137173a: Status 404 returned error can't find the container with id d4e65943d65a1bf4d073628766773372f3dfce9e09386a77810d6fbda137173a Jul 10 00:40:34.342938 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.342865 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6pkcd"] Jul 10 00:40:34.345241 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.345225 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.349730 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.349708 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 10 00:40:34.349872 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.349708 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zv2n7\"" Jul 10 00:40:34.356925 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.356901 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6pkcd"] Jul 10 00:40:34.360605 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.360586 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa267b4e-c553-4807-8665-4c86b50b1c65-plugins-config-volume\") pod \"payload-processing-58f86c7798-6pkcd\" (UID: \"aa267b4e-c553-4807-8665-4c86b50b1c65\") " pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.360700 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.360625 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khrcm\" (UniqueName: \"kubernetes.io/projected/aa267b4e-c553-4807-8665-4c86b50b1c65-kube-api-access-khrcm\") pod \"payload-processing-58f86c7798-6pkcd\" (UID: \"aa267b4e-c553-4807-8665-4c86b50b1c65\") " pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.461996 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.461964 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa267b4e-c553-4807-8665-4c86b50b1c65-plugins-config-volume\") pod \"payload-processing-58f86c7798-6pkcd\" (UID: \"aa267b4e-c553-4807-8665-4c86b50b1c65\") " pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.462169 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.462019 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khrcm\" (UniqueName: \"kubernetes.io/projected/aa267b4e-c553-4807-8665-4c86b50b1c65-kube-api-access-khrcm\") pod \"payload-processing-58f86c7798-6pkcd\" (UID: \"aa267b4e-c553-4807-8665-4c86b50b1c65\") " pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.462717 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.462691 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa267b4e-c553-4807-8665-4c86b50b1c65-plugins-config-volume\") pod \"payload-processing-58f86c7798-6pkcd\" (UID: \"aa267b4e-c553-4807-8665-4c86b50b1c65\") " pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.471179 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.471154 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khrcm\" (UniqueName: \"kubernetes.io/projected/aa267b4e-c553-4807-8665-4c86b50b1c65-kube-api-access-khrcm\") pod \"payload-processing-58f86c7798-6pkcd\" (UID: \"aa267b4e-c553-4807-8665-4c86b50b1c65\") " pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.655390 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.655298 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:34.778944 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.778916 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6pkcd"] Jul 10 00:40:34.781605 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:40:34.781579 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa267b4e_c553_4807_8665_4c86b50b1c65.slice/crio-69b8cfd549eb23235f8f17b9313a2f187fd9fa2983cf7fbd74e376c5124edc6e WatchSource:0}: Error finding container 69b8cfd549eb23235f8f17b9313a2f187fd9fa2983cf7fbd74e376c5124edc6e: Status 404 returned error can't find the container with id 69b8cfd549eb23235f8f17b9313a2f187fd9fa2983cf7fbd74e376c5124edc6e Jul 10 00:40:34.928271 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.928166 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" event={"ID":"aa267b4e-c553-4807-8665-4c86b50b1c65","Type":"ContainerStarted","Data":"69b8cfd549eb23235f8f17b9313a2f187fd9fa2983cf7fbd74e376c5124edc6e"} Jul 10 00:40:34.929315 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:34.929296 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" event={"ID":"c5a020d6-f9a5-4e0a-8844-93373f810970","Type":"ContainerStarted","Data":"d4e65943d65a1bf4d073628766773372f3dfce9e09386a77810d6fbda137173a"} Jul 10 00:40:36.939037 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:36.938999 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" event={"ID":"aa267b4e-c553-4807-8665-4c86b50b1c65","Type":"ContainerStarted","Data":"c4e031ead0d0b72ff9b70091479cae5ef7fee5b74f8d4cb12b053c5ab2b8b109"} Jul 10 00:40:36.939545 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:36.939146 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:40:36.940537 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:36.940513 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" event={"ID":"c5a020d6-f9a5-4e0a-8844-93373f810970","Type":"ContainerStarted","Data":"d324682ee57341cc0cc57b2a84136ec398f0ffa46a5b32fbbec216a1d59816b2"} Jul 10 00:40:36.940664 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:36.940598 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:36.954737 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:36.954585 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" podStartSLOduration=1.288607393 podStartE2EDuration="2.954568805s" totalImagesPullingTime="1.665961412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:40:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:40:36.953724856 +0000 UTC m=+857.396453918" watchObservedRunningTime="2026-07-10 00:40:36.954568805 +0000 UTC m=+857.397297855" Jul 10 00:40:36.969578 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:36.969535 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" podStartSLOduration=1.7413815320000001 podStartE2EDuration="3.969522649s" totalImagesPullingTime="2.228141117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:40:36.967858269 +0000 UTC m=+857.410587316" watchObservedRunningTime="2026-07-10 00:40:36.969522649 +0000 UTC m=+857.412251698" Jul 10 00:40:42.949688 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:42.949660 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c5447fb99-fcplc" Jul 10 00:40:47.946234 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:40:47.946208 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-6pkcd" Jul 10 00:41:01.181208 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.181180 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7"] Jul 10 00:41:01.184315 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.184297 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.186650 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.186629 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 00:41:01.187443 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.187426 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 10 00:41:01.187574 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.187429 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bqvgx\"" Jul 10 00:41:01.187574 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.187501 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 00:41:01.195450 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.195428 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7"] Jul 10 00:41:01.268320 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.268281 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.268501 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.268341 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.268501 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.268409 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.268501 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.268464 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfgz8\" (UniqueName: \"kubernetes.io/projected/9d177344-ce21-427b-b37d-aebab975a7bd-kube-api-access-lfgz8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.268501 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.268489 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d177344-ce21-427b-b37d-aebab975a7bd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.268668 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.268518 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.369618 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.369578 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.369819 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.369628 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.369819 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.369668 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfgz8\" (UniqueName: \"kubernetes.io/projected/9d177344-ce21-427b-b37d-aebab975a7bd-kube-api-access-lfgz8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.369819 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.369695 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d177344-ce21-427b-b37d-aebab975a7bd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.369819 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.369714 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.369819 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.369748 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.370055 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.370035 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.370111 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.370081 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.370164 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.370151 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.372150 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.372121 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d177344-ce21-427b-b37d-aebab975a7bd-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.372386 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.372369 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d177344-ce21-427b-b37d-aebab975a7bd-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.379368 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.379347 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfgz8\" (UniqueName: \"kubernetes.io/projected/9d177344-ce21-427b-b37d-aebab975a7bd-kube-api-access-lfgz8\") pod \"e2e-distinct-simulated-kserve-59975d69f9-28hf7\" (UID: \"9d177344-ce21-427b-b37d-aebab975a7bd\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.494378 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.494299 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:01.619789 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:01.619765 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7"] Jul 10 00:41:01.622340 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:41:01.622314 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d177344_ce21_427b_b37d_aebab975a7bd.slice/crio-08e045159cf49b2c7b64200c9dac90b7bd9b0e5e8313ea537ad2dc8b632cfe92 WatchSource:0}: Error finding container 08e045159cf49b2c7b64200c9dac90b7bd9b0e5e8313ea537ad2dc8b632cfe92: Status 404 returned error can't find the container with id 08e045159cf49b2c7b64200c9dac90b7bd9b0e5e8313ea537ad2dc8b632cfe92 Jul 10 00:41:02.034391 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:02.034353 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" event={"ID":"9d177344-ce21-427b-b37d-aebab975a7bd","Type":"ContainerStarted","Data":"08e045159cf49b2c7b64200c9dac90b7bd9b0e5e8313ea537ad2dc8b632cfe92"} Jul 10 00:41:03.040100 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:03.040004 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" event={"ID":"9d177344-ce21-427b-b37d-aebab975a7bd","Type":"ContainerStarted","Data":"bb7f844fbc0776067358c0b5636efded06ac60f2f511e37062a306369df05b82"} Jul 10 00:41:03.058633 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:03.058572 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" podStartSLOduration=1.091740482 podStartE2EDuration="2.05855265s" totalImagesPullingTime="966.812168ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:41:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:41:03.057658611 +0000 UTC m=+883.500387660" watchObservedRunningTime="2026-07-10 00:41:03.05855265 +0000 UTC m=+883.501281700" Jul 10 00:41:11.495481 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:11.495372 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:11.495481 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:11.495443 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:11.507790 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:11.507768 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:12.085540 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:12.085512 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-28hf7" Jul 10 00:41:19.954661 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:19.954636 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:41:19.956170 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:19.956147 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:41:54.480935 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.480908 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq"] Jul 10 00:41:54.483900 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.483883 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.488169 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.488150 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 00:41:54.506646 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.506617 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq"] Jul 10 00:41:54.573428 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.573390 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.573575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.573441 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.573575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.573457 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.573575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.573490 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlw4x\" (UniqueName: \"kubernetes.io/projected/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-kube-api-access-qlw4x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.573575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.573530 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.573575 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.573563 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.673985 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.673958 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674158 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674007 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674158 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674041 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674158 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674057 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlw4x\" (UniqueName: \"kubernetes.io/projected/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-kube-api-access-qlw4x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674158 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674077 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674371 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674168 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674449 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674427 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674504 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674432 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.674555 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.674532 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.676391 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.676374 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.676711 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.676695 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.683322 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.683302 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlw4x\" (UniqueName: \"kubernetes.io/projected/48d42cda-d7e3-4e44-b6bb-1d6d41b40965-kube-api-access-qlw4x\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-d5rpq\" (UID: \"48d42cda-d7e3-4e44-b6bb-1d6d41b40965\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.792904 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.792817 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:41:54.926176 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:54.926154 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq"] Jul 10 00:41:54.928537 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:41:54.928511 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d42cda_d7e3_4e44_b6bb_1d6d41b40965.slice/crio-7436f4f8ba1c3041ee41353c88edf2926133fee5797ad82ac6037a2718f40c9b WatchSource:0}: Error finding container 7436f4f8ba1c3041ee41353c88edf2926133fee5797ad82ac6037a2718f40c9b: Status 404 returned error can't find the container with id 7436f4f8ba1c3041ee41353c88edf2926133fee5797ad82ac6037a2718f40c9b Jul 10 00:41:55.232225 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:55.232190 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" event={"ID":"48d42cda-d7e3-4e44-b6bb-1d6d41b40965","Type":"ContainerStarted","Data":"bd2f5280bedf8ae7584d09fb2be137d25ba735b134098cb599131e349875079a"} Jul 10 00:41:55.232395 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:55.232267 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" event={"ID":"48d42cda-d7e3-4e44-b6bb-1d6d41b40965","Type":"ContainerStarted","Data":"7436f4f8ba1c3041ee41353c88edf2926133fee5797ad82ac6037a2718f40c9b"} Jul 10 00:41:55.268678 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:41:55.268626 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" podStartSLOduration=1.111143035 podStartE2EDuration="1.268609439s" totalImagesPullingTime="157.466404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:41:55.265647056 +0000 UTC m=+935.708376107" watchObservedRunningTime="2026-07-10 00:41:55.268609439 +0000 UTC m=+935.711338489" Jul 10 00:42:04.793558 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:42:04.793521 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:42:04.794073 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:42:04.793604 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:42:04.806027 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:42:04.806006 2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:42:05.283367 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:42:05.283340 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-d5rpq" Jul 10 00:46:19.981894 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:46:19.981815 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:46:19.983972 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:46:19.983953 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:51:20.003666 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:51:20.003636 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:51:20.006694 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:51:20.006672 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:56:20.028449 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:20.028419 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:56:20.033627 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:20.033603 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 00:56:51.791131 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.791091 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz"] Jul 10 00:56:51.794300 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.794283 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.796591 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.796566 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-af194e18-serving-cert\"" Jul 10 00:56:51.805270 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.805221 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz"] Jul 10 00:56:51.879341 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.879300 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqm79\" (UniqueName: \"kubernetes.io/projected/53e33876-8b2b-45a6-ab47-0db45b7456fe-kube-api-access-mqm79\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.879521 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.879365 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e33876-8b2b-45a6-ab47-0db45b7456fe-openshift-service-ca\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.879521 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.879443 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e33876-8b2b-45a6-ab47-0db45b7456fe-maas-api-tls\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.980226 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.980190 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e33876-8b2b-45a6-ab47-0db45b7456fe-openshift-service-ca\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.980422 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.980240 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e33876-8b2b-45a6-ab47-0db45b7456fe-maas-api-tls\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.980422 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.980307 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqm79\" (UniqueName: \"kubernetes.io/projected/53e33876-8b2b-45a6-ab47-0db45b7456fe-kube-api-access-mqm79\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.980865 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.980833 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e33876-8b2b-45a6-ab47-0db45b7456fe-openshift-service-ca\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:51.982868 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:51.982844 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e33876-8b2b-45a6-ab47-0db45b7456fe-maas-api-tls\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:52.002643 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.002608 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqm79\" (UniqueName: \"kubernetes.io/projected/53e33876-8b2b-45a6-ab47-0db45b7456fe-kube-api-access-mqm79\") pod \"maas-api-e2e-ait-af194e18-54955ff4cf-426bz\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:52.105401 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.105359 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:52.245319 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.245280 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz"] Jul 10 00:56:52.249213 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:56:52.249187 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e33876_8b2b_45a6_ab47_0db45b7456fe.slice/crio-060f44d21db01a2133d2a3db4a411af76d488c6bcbec276644cc9091c3fd955b WatchSource:0}: Error finding container 060f44d21db01a2133d2a3db4a411af76d488c6bcbec276644cc9091c3fd955b: Status 404 returned error can't find the container with id 060f44d21db01a2133d2a3db4a411af76d488c6bcbec276644cc9091c3fd955b Jul 10 00:56:52.250560 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.250541 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 00:56:52.791075 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.791038 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" event={"ID":"53e33876-8b2b-45a6-ab47-0db45b7456fe","Type":"ContainerStarted","Data":"f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3"} Jul 10 00:56:52.791303 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.791088 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" event={"ID":"53e33876-8b2b-45a6-ab47-0db45b7456fe","Type":"ContainerStarted","Data":"060f44d21db01a2133d2a3db4a411af76d488c6bcbec276644cc9091c3fd955b"} Jul 10 00:56:52.791303 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.791178 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:52.807024 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:52.806971 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" podStartSLOduration=1.430416583 podStartE2EDuration="1.806953357s" totalImagesPullingTime="376.536774ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:56:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:56:52.805677382 +0000 UTC m=+1833.248406435" watchObservedRunningTime="2026-07-10 00:56:52.806953357 +0000 UTC m=+1833.249682407" Jul 10 00:56:57.094583 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.094544 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz"] Jul 10 00:56:57.095128 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.094718 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" podUID="53e33876-8b2b-45a6-ab47-0db45b7456fe" containerName="maas-api" containerID="cri-o://f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3" gracePeriod=30 Jul 10 00:56:57.101026 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.101003 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:57.338723 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.338695 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:57.428328 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.428215 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqm79\" (UniqueName: \"kubernetes.io/projected/53e33876-8b2b-45a6-ab47-0db45b7456fe-kube-api-access-mqm79\") pod \"53e33876-8b2b-45a6-ab47-0db45b7456fe\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " Jul 10 00:56:57.428328 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.428285 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e33876-8b2b-45a6-ab47-0db45b7456fe-openshift-service-ca\") pod \"53e33876-8b2b-45a6-ab47-0db45b7456fe\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " Jul 10 00:56:57.428328 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.428317 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e33876-8b2b-45a6-ab47-0db45b7456fe-maas-api-tls\") pod \"53e33876-8b2b-45a6-ab47-0db45b7456fe\" (UID: \"53e33876-8b2b-45a6-ab47-0db45b7456fe\") " Jul 10 00:56:57.428609 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.428585 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53e33876-8b2b-45a6-ab47-0db45b7456fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "53e33876-8b2b-45a6-ab47-0db45b7456fe" (UID: "53e33876-8b2b-45a6-ab47-0db45b7456fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:56:57.430585 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.430554 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53e33876-8b2b-45a6-ab47-0db45b7456fe-kube-api-access-mqm79" (OuterVolumeSpecName: "kube-api-access-mqm79") pod "53e33876-8b2b-45a6-ab47-0db45b7456fe" (UID: "53e33876-8b2b-45a6-ab47-0db45b7456fe"). InnerVolumeSpecName "kube-api-access-mqm79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:56:57.430694 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.430595 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53e33876-8b2b-45a6-ab47-0db45b7456fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "53e33876-8b2b-45a6-ab47-0db45b7456fe" (UID: "53e33876-8b2b-45a6-ab47-0db45b7456fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:56:57.528939 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.528901 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqm79\" (UniqueName: \"kubernetes.io/projected/53e33876-8b2b-45a6-ab47-0db45b7456fe-kube-api-access-mqm79\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:56:57.528939 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.528929 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/53e33876-8b2b-45a6-ab47-0db45b7456fe-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:56:57.528939 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.528942 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/53e33876-8b2b-45a6-ab47-0db45b7456fe-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:56:57.813557 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.813463 2579 generic.go:356] "Generic (PLEG): container finished" podID="53e33876-8b2b-45a6-ab47-0db45b7456fe" containerID="f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3" exitCode=0 Jul 10 00:56:57.813557 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.813517 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" Jul 10 00:56:57.813557 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.813540 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" event={"ID":"53e33876-8b2b-45a6-ab47-0db45b7456fe","Type":"ContainerDied","Data":"f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3"} Jul 10 00:56:57.813797 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.813591 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz" event={"ID":"53e33876-8b2b-45a6-ab47-0db45b7456fe","Type":"ContainerDied","Data":"060f44d21db01a2133d2a3db4a411af76d488c6bcbec276644cc9091c3fd955b"} Jul 10 00:56:57.813797 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.813620 2579 scope.go:117] "RemoveContainer" containerID="f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3" Jul 10 00:56:57.822409 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.822393 2579 scope.go:117] "RemoveContainer" containerID="f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3" Jul 10 00:56:57.822641 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:56:57.822624 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3\": container with ID starting with f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3 not found: ID does not exist" containerID="f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3" Jul 10 00:56:57.822700 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.822649 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3"} err="failed to get container status \"f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3\": rpc error: code = NotFound desc = could not find container \"f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3\": container with ID starting with f0aef0eaed689a6ad698a08aaf59621ecbd105eeb417ae49892a69a8e8196cd3 not found: ID does not exist" Jul 10 00:56:57.833310 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.833286 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz"] Jul 10 00:56:57.837779 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:57.837759 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-af194e18-54955ff4cf-426bz"] Jul 10 00:56:58.026500 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:56:58.026466 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53e33876-8b2b-45a6-ab47-0db45b7456fe" path="/var/lib/kubelet/pods/53e33876-8b2b-45a6-ab47-0db45b7456fe/volumes" Jul 10 00:57:04.055555 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.055521 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf"] Jul 10 00:57:04.055947 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.055933 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53e33876-8b2b-45a6-ab47-0db45b7456fe" containerName="maas-api" Jul 10 00:57:04.055987 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.055949 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="53e33876-8b2b-45a6-ab47-0db45b7456fe" containerName="maas-api" Jul 10 00:57:04.056031 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.056019 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="53e33876-8b2b-45a6-ab47-0db45b7456fe" containerName="maas-api" Jul 10 00:57:04.060419 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.060400 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.062670 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.062646 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a3da50c4-serving-cert\"" Jul 10 00:57:04.069139 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.069119 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf"] Jul 10 00:57:04.186039 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.186003 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3a439f-1a22-4227-889e-a8779c014ce7-openshift-service-ca\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.186211 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.186057 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3a439f-1a22-4227-889e-a8779c014ce7-maas-api-tls\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.186211 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.186103 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzprq\" (UniqueName: \"kubernetes.io/projected/ce3a439f-1a22-4227-889e-a8779c014ce7-kube-api-access-jzprq\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.286709 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.286666 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3a439f-1a22-4227-889e-a8779c014ce7-openshift-service-ca\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.286886 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.286720 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3a439f-1a22-4227-889e-a8779c014ce7-maas-api-tls\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.286886 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.286752 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzprq\" (UniqueName: \"kubernetes.io/projected/ce3a439f-1a22-4227-889e-a8779c014ce7-kube-api-access-jzprq\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.287349 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.287329 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3a439f-1a22-4227-889e-a8779c014ce7-openshift-service-ca\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.289274 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.289244 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3a439f-1a22-4227-889e-a8779c014ce7-maas-api-tls\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.297220 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.297189 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzprq\" (UniqueName: \"kubernetes.io/projected/ce3a439f-1a22-4227-889e-a8779c014ce7-kube-api-access-jzprq\") pod \"maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.371290 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.371213 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:04.504685 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.504651 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf"] Jul 10 00:57:04.507741 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:57:04.507706 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3a439f_1a22_4227_889e_a8779c014ce7.slice/crio-39b8a3d93e10824d9283f4bf9bdfa0917c800d9e1358d929e4c5b36073ffff5f WatchSource:0}: Error finding container 39b8a3d93e10824d9283f4bf9bdfa0917c800d9e1358d929e4c5b36073ffff5f: Status 404 returned error can't find the container with id 39b8a3d93e10824d9283f4bf9bdfa0917c800d9e1358d929e4c5b36073ffff5f Jul 10 00:57:04.844267 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:04.844221 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" event={"ID":"ce3a439f-1a22-4227-889e-a8779c014ce7","Type":"ContainerStarted","Data":"39b8a3d93e10824d9283f4bf9bdfa0917c800d9e1358d929e4c5b36073ffff5f"} Jul 10 00:57:05.849635 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:05.849594 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" event={"ID":"ce3a439f-1a22-4227-889e-a8779c014ce7","Type":"ContainerStarted","Data":"e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8"} Jul 10 00:57:05.850072 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:05.849714 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:05.864883 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:05.864829 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" podStartSLOduration=1.505833583 podStartE2EDuration="1.864813489s" totalImagesPullingTime="358.979906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:57:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:57:05.864233925 +0000 UTC m=+1846.306962976" watchObservedRunningTime="2026-07-10 00:57:05.864813489 +0000 UTC m=+1846.307542540" Jul 10 00:57:09.430062 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.430028 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf"] Jul 10 00:57:09.430509 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.430268 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" podUID="ce3a439f-1a22-4227-889e-a8779c014ce7" containerName="maas-api" containerID="cri-o://e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8" gracePeriod=30 Jul 10 00:57:09.435965 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.435944 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:09.668861 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.668837 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:09.835725 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.835698 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3a439f-1a22-4227-889e-a8779c014ce7-openshift-service-ca\") pod \"ce3a439f-1a22-4227-889e-a8779c014ce7\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " Jul 10 00:57:09.835909 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.835743 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzprq\" (UniqueName: \"kubernetes.io/projected/ce3a439f-1a22-4227-889e-a8779c014ce7-kube-api-access-jzprq\") pod \"ce3a439f-1a22-4227-889e-a8779c014ce7\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " Jul 10 00:57:09.835909 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.835784 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3a439f-1a22-4227-889e-a8779c014ce7-maas-api-tls\") pod \"ce3a439f-1a22-4227-889e-a8779c014ce7\" (UID: \"ce3a439f-1a22-4227-889e-a8779c014ce7\") " Jul 10 00:57:09.836103 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.836077 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce3a439f-1a22-4227-889e-a8779c014ce7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ce3a439f-1a22-4227-889e-a8779c014ce7" (UID: "ce3a439f-1a22-4227-889e-a8779c014ce7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:57:09.838036 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.837996 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce3a439f-1a22-4227-889e-a8779c014ce7-kube-api-access-jzprq" (OuterVolumeSpecName: "kube-api-access-jzprq") pod "ce3a439f-1a22-4227-889e-a8779c014ce7" (UID: "ce3a439f-1a22-4227-889e-a8779c014ce7"). InnerVolumeSpecName "kube-api-access-jzprq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:57:09.838132 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.838084 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce3a439f-1a22-4227-889e-a8779c014ce7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ce3a439f-1a22-4227-889e-a8779c014ce7" (UID: "ce3a439f-1a22-4227-889e-a8779c014ce7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:57:09.866908 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.866874 2579 generic.go:356] "Generic (PLEG): container finished" podID="ce3a439f-1a22-4227-889e-a8779c014ce7" containerID="e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8" exitCode=0 Jul 10 00:57:09.867067 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.866920 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" Jul 10 00:57:09.867067 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.866961 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" event={"ID":"ce3a439f-1a22-4227-889e-a8779c014ce7","Type":"ContainerDied","Data":"e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8"} Jul 10 00:57:09.867067 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.867015 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf" event={"ID":"ce3a439f-1a22-4227-889e-a8779c014ce7","Type":"ContainerDied","Data":"39b8a3d93e10824d9283f4bf9bdfa0917c800d9e1358d929e4c5b36073ffff5f"} Jul 10 00:57:09.867067 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.867044 2579 scope.go:117] "RemoveContainer" containerID="e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8" Jul 10 00:57:09.875599 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.875566 2579 scope.go:117] "RemoveContainer" containerID="e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8" Jul 10 00:57:09.875836 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:57:09.875818 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8\": container with ID starting with e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8 not found: ID does not exist" containerID="e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8" Jul 10 00:57:09.875877 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.875844 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8"} err="failed to get container status \"e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8\": rpc error: code = NotFound desc = could not find container \"e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8\": container with ID starting with e199d641ebe3b768343b5117c4fab1cfb2084937c5d758e3610f02c5f1b4d5b8 not found: ID does not exist" Jul 10 00:57:09.889389 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.889362 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf"] Jul 10 00:57:09.892764 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.892743 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a3da50c4-6964f4b96f-8qlgf"] Jul 10 00:57:09.936587 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.936553 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzprq\" (UniqueName: \"kubernetes.io/projected/ce3a439f-1a22-4227-889e-a8779c014ce7-kube-api-access-jzprq\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:57:09.936587 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.936580 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3a439f-1a22-4227-889e-a8779c014ce7-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:57:09.936587 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:09.936591 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3a439f-1a22-4227-889e-a8779c014ce7-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:57:10.025766 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:10.025724 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce3a439f-1a22-4227-889e-a8779c014ce7" path="/var/lib/kubelet/pods/ce3a439f-1a22-4227-889e-a8779c014ce7/volumes" Jul 10 00:57:22.689162 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.689124 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c"] Jul 10 00:57:22.689682 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.689581 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce3a439f-1a22-4227-889e-a8779c014ce7" containerName="maas-api" Jul 10 00:57:22.689682 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.689597 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce3a439f-1a22-4227-889e-a8779c014ce7" containerName="maas-api" Jul 10 00:57:22.689765 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.689711 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce3a439f-1a22-4227-889e-a8779c014ce7" containerName="maas-api" Jul 10 00:57:22.692840 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.692821 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.695416 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.695381 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-a1ce34f1-serving-cert\"" Jul 10 00:57:22.703833 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.703808 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c"] Jul 10 00:57:22.846632 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.846599 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9ft8\" (UniqueName: \"kubernetes.io/projected/e7aabfab-8510-4b6f-845e-259b09a64490-kube-api-access-r9ft8\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.846822 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.846647 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.846822 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.846703 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7aabfab-8510-4b6f-845e-259b09a64490-openshift-service-ca\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.947835 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.947735 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7aabfab-8510-4b6f-845e-259b09a64490-openshift-service-ca\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.947835 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.947811 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9ft8\" (UniqueName: \"kubernetes.io/projected/e7aabfab-8510-4b6f-845e-259b09a64490-kube-api-access-r9ft8\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.947835 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.947832 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.948110 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:57:22.947924 2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-serving-cert: secret "maas-api-e2e-derive-a1ce34f1-serving-cert" not found Jul 10 00:57:22.948110 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:57:22.947982 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls podName:e7aabfab-8510-4b6f-845e-259b09a64490 nodeName:}" failed. No retries permitted until 2026-07-10 00:57:23.44796719 +0000 UTC m=+1863.890696219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls") pod "maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" (UID: "e7aabfab-8510-4b6f-845e-259b09a64490") : secret "maas-api-e2e-derive-a1ce34f1-serving-cert" not found Jul 10 00:57:22.948519 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.948497 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7aabfab-8510-4b6f-845e-259b09a64490-openshift-service-ca\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:22.956377 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:22.956356 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9ft8\" (UniqueName: \"kubernetes.io/projected/e7aabfab-8510-4b6f-845e-259b09a64490-kube-api-access-r9ft8\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:23.451907 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:23.451865 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:23.454549 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:23.454524 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls\") pod \"maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:23.603445 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:23.603409 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:23.731128 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:23.730941 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c"] Jul 10 00:57:23.733956 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:57:23.733926 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7aabfab_8510_4b6f_845e_259b09a64490.slice/crio-30e638b3709378bb4cacb5bb74d2e9c5d12cd969442d519fa85ff015d1bc99d8 WatchSource:0}: Error finding container 30e638b3709378bb4cacb5bb74d2e9c5d12cd969442d519fa85ff015d1bc99d8: Status 404 returned error can't find the container with id 30e638b3709378bb4cacb5bb74d2e9c5d12cd969442d519fa85ff015d1bc99d8 Jul 10 00:57:23.923936 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:23.923902 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" event={"ID":"e7aabfab-8510-4b6f-845e-259b09a64490","Type":"ContainerStarted","Data":"30e638b3709378bb4cacb5bb74d2e9c5d12cd969442d519fa85ff015d1bc99d8"} Jul 10 00:57:24.929500 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:24.929468 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" event={"ID":"e7aabfab-8510-4b6f-845e-259b09a64490","Type":"ContainerStarted","Data":"6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102"} Jul 10 00:57:24.929918 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:24.929571 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:24.945717 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:24.945670 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" podStartSLOduration=2.543430443 podStartE2EDuration="2.945655173s" totalImagesPullingTime="402.22473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:57:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:57:24.944807116 +0000 UTC m=+1865.387536172" watchObservedRunningTime="2026-07-10 00:57:24.945655173 +0000 UTC m=+1865.388384223" Jul 10 00:57:27.476989 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.476950 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c"] Jul 10 00:57:27.477408 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.477181 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" podUID="e7aabfab-8510-4b6f-845e-259b09a64490" containerName="maas-api" containerID="cri-o://6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102" gracePeriod=30 Jul 10 00:57:27.717333 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.717309 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:27.785886 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.785802 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7aabfab-8510-4b6f-845e-259b09a64490-openshift-service-ca\") pod \"e7aabfab-8510-4b6f-845e-259b09a64490\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " Jul 10 00:57:27.785886 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.785863 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9ft8\" (UniqueName: \"kubernetes.io/projected/e7aabfab-8510-4b6f-845e-259b09a64490-kube-api-access-r9ft8\") pod \"e7aabfab-8510-4b6f-845e-259b09a64490\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " Jul 10 00:57:27.786069 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.785927 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls\") pod \"e7aabfab-8510-4b6f-845e-259b09a64490\" (UID: \"e7aabfab-8510-4b6f-845e-259b09a64490\") " Jul 10 00:57:27.786202 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.786182 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7aabfab-8510-4b6f-845e-259b09a64490-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7aabfab-8510-4b6f-845e-259b09a64490" (UID: "e7aabfab-8510-4b6f-845e-259b09a64490"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:57:27.788041 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.788007 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7aabfab-8510-4b6f-845e-259b09a64490" (UID: "e7aabfab-8510-4b6f-845e-259b09a64490"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:57:27.788145 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.788110 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7aabfab-8510-4b6f-845e-259b09a64490-kube-api-access-r9ft8" (OuterVolumeSpecName: "kube-api-access-r9ft8") pod "e7aabfab-8510-4b6f-845e-259b09a64490" (UID: "e7aabfab-8510-4b6f-845e-259b09a64490"). InnerVolumeSpecName "kube-api-access-r9ft8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:57:27.886940 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.886909 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7aabfab-8510-4b6f-845e-259b09a64490-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:57:27.886940 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.886933 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9ft8\" (UniqueName: \"kubernetes.io/projected/e7aabfab-8510-4b6f-845e-259b09a64490-kube-api-access-r9ft8\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:57:27.886940 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.886944 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7aabfab-8510-4b6f-845e-259b09a64490-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:57:27.943563 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.943531 2579 generic.go:356] "Generic (PLEG): container finished" podID="e7aabfab-8510-4b6f-845e-259b09a64490" containerID="6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102" exitCode=0 Jul 10 00:57:27.943719 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.943581 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" Jul 10 00:57:27.943719 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.943601 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" event={"ID":"e7aabfab-8510-4b6f-845e-259b09a64490","Type":"ContainerDied","Data":"6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102"} Jul 10 00:57:27.943719 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.943646 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c" event={"ID":"e7aabfab-8510-4b6f-845e-259b09a64490","Type":"ContainerDied","Data":"30e638b3709378bb4cacb5bb74d2e9c5d12cd969442d519fa85ff015d1bc99d8"} Jul 10 00:57:27.943719 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.943667 2579 scope.go:117] "RemoveContainer" containerID="6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102" Jul 10 00:57:27.952279 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.952240 2579 scope.go:117] "RemoveContainer" containerID="6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102" Jul 10 00:57:27.952544 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:57:27.952526 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102\": container with ID starting with 6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102 not found: ID does not exist" containerID="6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102" Jul 10 00:57:27.952611 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.952557 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102"} err="failed to get container status \"6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102\": rpc error: code = NotFound desc = could not find container \"6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102\": container with ID starting with 6c6ed8e70d1c0d54ece143658a3693835d54a45147e6bf2abfc62b3a2565f102 not found: ID does not exist" Jul 10 00:57:27.963940 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.963912 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c"] Jul 10 00:57:27.967807 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:27.967783 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-a1ce34f1-75d4458bb5-qxg8c"] Jul 10 00:57:28.025913 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:57:28.025881 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7aabfab-8510-4b6f-845e-259b09a64490" path="/var/lib/kubelet/pods/e7aabfab-8510-4b6f-845e-259b09a64490/volumes" Jul 10 00:59:12.301600 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.301516 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p"] Jul 10 00:59:12.302145 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.301925 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7aabfab-8510-4b6f-845e-259b09a64490" containerName="maas-api" Jul 10 00:59:12.302145 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.301942 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7aabfab-8510-4b6f-845e-259b09a64490" containerName="maas-api" Jul 10 00:59:12.302145 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.302030 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7aabfab-8510-4b6f-845e-259b09a64490" containerName="maas-api" Jul 10 00:59:12.304943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.304921 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.307374 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.307352 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-84296f95-serving-cert\"" Jul 10 00:59:12.314726 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.314701 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p"] Jul 10 00:59:12.440613 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.440576 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg6gc\" (UniqueName: \"kubernetes.io/projected/e26401d0-683c-4186-873f-df989aac2aca-kube-api-access-dg6gc\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.440802 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.440703 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26401d0-683c-4186-873f-df989aac2aca-openshift-service-ca\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.440802 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.440754 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.541326 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.541292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg6gc\" (UniqueName: \"kubernetes.io/projected/e26401d0-683c-4186-873f-df989aac2aca-kube-api-access-dg6gc\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.541481 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.541352 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26401d0-683c-4186-873f-df989aac2aca-openshift-service-ca\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.541481 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.541380 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.541481 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:59:12.541460 2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-serving-cert: secret "maas-api-e2e-mt-84296f95-serving-cert" not found Jul 10 00:59:12.541603 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:59:12.541515 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls podName:e26401d0-683c-4186-873f-df989aac2aca nodeName:}" failed. No retries permitted until 2026-07-10 00:59:13.041498278 +0000 UTC m=+1973.484227306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls") pod "maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" (UID: "e26401d0-683c-4186-873f-df989aac2aca") : secret "maas-api-e2e-mt-84296f95-serving-cert" not found Jul 10 00:59:12.541962 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.541943 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26401d0-683c-4186-873f-df989aac2aca-openshift-service-ca\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:12.551666 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:12.551608 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg6gc\" (UniqueName: \"kubernetes.io/projected/e26401d0-683c-4186-873f-df989aac2aca-kube-api-access-dg6gc\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:13.045868 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:13.045831 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:13.048400 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:13.048376 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls\") pod \"maas-api-e2e-mt-84296f95-74b5cf94-lgk4p\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:13.214848 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:13.214815 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:13.345422 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:13.345363 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p"] Jul 10 00:59:13.348458 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:59:13.348431 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26401d0_683c_4186_873f_df989aac2aca.slice/crio-1af978b2cf2325e068eca329ed2f5748ebd1ed4a77c68398799976793fef3647 WatchSource:0}: Error finding container 1af978b2cf2325e068eca329ed2f5748ebd1ed4a77c68398799976793fef3647: Status 404 returned error can't find the container with id 1af978b2cf2325e068eca329ed2f5748ebd1ed4a77c68398799976793fef3647 Jul 10 00:59:13.353996 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:13.353965 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" event={"ID":"e26401d0-683c-4186-873f-df989aac2aca","Type":"ContainerStarted","Data":"1af978b2cf2325e068eca329ed2f5748ebd1ed4a77c68398799976793fef3647"} Jul 10 00:59:14.359654 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:14.359617 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" event={"ID":"e26401d0-683c-4186-873f-df989aac2aca","Type":"ContainerStarted","Data":"e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196"} Jul 10 00:59:14.360039 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:14.359689 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:14.376395 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:14.376347 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" podStartSLOduration=1.9346626260000002 podStartE2EDuration="2.376330731s" totalImagesPullingTime="441.668105ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:59:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:59:14.374507484 +0000 UTC m=+1974.817236534" watchObservedRunningTime="2026-07-10 00:59:14.376330731 +0000 UTC m=+1974.819059822" Jul 10 00:59:20.368943 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:20.368916 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:24.060330 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.060299 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p"] Jul 10 00:59:24.060880 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.060495 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" podUID="e26401d0-683c-4186-873f-df989aac2aca" containerName="maas-api" containerID="cri-o://e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196" gracePeriod=30 Jul 10 00:59:24.308859 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.308836 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:24.331441 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.331418 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg6gc\" (UniqueName: \"kubernetes.io/projected/e26401d0-683c-4186-873f-df989aac2aca-kube-api-access-dg6gc\") pod \"e26401d0-683c-4186-873f-df989aac2aca\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " Jul 10 00:59:24.331570 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.331448 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26401d0-683c-4186-873f-df989aac2aca-openshift-service-ca\") pod \"e26401d0-683c-4186-873f-df989aac2aca\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " Jul 10 00:59:24.331570 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.331497 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls\") pod \"e26401d0-683c-4186-873f-df989aac2aca\" (UID: \"e26401d0-683c-4186-873f-df989aac2aca\") " Jul 10 00:59:24.331877 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.331851 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e26401d0-683c-4186-873f-df989aac2aca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e26401d0-683c-4186-873f-df989aac2aca" (UID: "e26401d0-683c-4186-873f-df989aac2aca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 00:59:24.333661 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.333641 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e26401d0-683c-4186-873f-df989aac2aca" (UID: "e26401d0-683c-4186-873f-df989aac2aca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 00:59:24.333739 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.333695 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e26401d0-683c-4186-873f-df989aac2aca-kube-api-access-dg6gc" (OuterVolumeSpecName: "kube-api-access-dg6gc") pod "e26401d0-683c-4186-873f-df989aac2aca" (UID: "e26401d0-683c-4186-873f-df989aac2aca"). InnerVolumeSpecName "kube-api-access-dg6gc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 00:59:24.398570 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.398538 2579 generic.go:356] "Generic (PLEG): container finished" podID="e26401d0-683c-4186-873f-df989aac2aca" containerID="e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196" exitCode=0 Jul 10 00:59:24.398724 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.398578 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" event={"ID":"e26401d0-683c-4186-873f-df989aac2aca","Type":"ContainerDied","Data":"e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196"} Jul 10 00:59:24.398724 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.398592 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" Jul 10 00:59:24.398724 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.398630 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p" event={"ID":"e26401d0-683c-4186-873f-df989aac2aca","Type":"ContainerDied","Data":"1af978b2cf2325e068eca329ed2f5748ebd1ed4a77c68398799976793fef3647"} Jul 10 00:59:24.398724 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.398655 2579 scope.go:117] "RemoveContainer" containerID="e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196" Jul 10 00:59:24.407075 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.407052 2579 scope.go:117] "RemoveContainer" containerID="e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196" Jul 10 00:59:24.407339 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:59:24.407319 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196\": container with ID starting with e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196 not found: ID does not exist" containerID="e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196" Jul 10 00:59:24.407425 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.407348 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196"} err="failed to get container status \"e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196\": rpc error: code = NotFound desc = could not find container \"e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196\": container with ID starting with e417c800feec4659ee7f30b89de7a95be4f0a76aa00cb5f62c50bca6cff15196 not found: ID does not exist" Jul 10 00:59:24.418524 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.418501 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p"] Jul 10 00:59:24.422877 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.422856 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-84296f95-74b5cf94-lgk4p"] Jul 10 00:59:24.431936 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.431915 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg6gc\" (UniqueName: \"kubernetes.io/projected/e26401d0-683c-4186-873f-df989aac2aca-kube-api-access-dg6gc\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:59:24.431936 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.431935 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e26401d0-683c-4186-873f-df989aac2aca-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:59:24.432051 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:24.431945 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e26401d0-683c-4186-873f-df989aac2aca-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 00:59:26.026554 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:26.026522 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e26401d0-683c-4186-873f-df989aac2aca" path="/var/lib/kubelet/pods/e26401d0-683c-4186-873f-df989aac2aca/volumes" Jul 10 00:59:52.626728 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.626698 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk"] Jul 10 00:59:52.627179 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.627032 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e26401d0-683c-4186-873f-df989aac2aca" containerName="maas-api" Jul 10 00:59:52.627179 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.627044 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e26401d0-683c-4186-873f-df989aac2aca" containerName="maas-api" Jul 10 00:59:52.627179 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.627093 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e26401d0-683c-4186-873f-df989aac2aca" containerName="maas-api" Jul 10 00:59:52.629930 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.629912 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.632326 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.632303 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-31899000-serving-cert\"" Jul 10 00:59:52.640290 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.640265 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk"] Jul 10 00:59:52.760117 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.760086 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051eba95-7161-4990-9954-0c80f59c8eea-openshift-service-ca\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.760321 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.760145 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgq57\" (UniqueName: \"kubernetes.io/projected/051eba95-7161-4990-9954-0c80f59c8eea-kube-api-access-jgq57\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.760321 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.760218 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.861561 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.861511 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.861742 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.861575 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051eba95-7161-4990-9954-0c80f59c8eea-openshift-service-ca\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.861742 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.861631 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgq57\" (UniqueName: \"kubernetes.io/projected/051eba95-7161-4990-9954-0c80f59c8eea-kube-api-access-jgq57\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.861742 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:59:52.861666 2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-31899000-serving-cert: secret "maas-api-e2e-mt-31899000-serving-cert" not found Jul 10 00:59:52.861742 ip-10-0-131-148 kubenswrapper[2579]: E0710 00:59:52.861740 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls podName:051eba95-7161-4990-9954-0c80f59c8eea nodeName:}" failed. No retries permitted until 2026-07-10 00:59:53.361722634 +0000 UTC m=+2013.804451662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls") pod "maas-api-e2e-mt-31899000-9d5fd6585-dqptk" (UID: "051eba95-7161-4990-9954-0c80f59c8eea") : secret "maas-api-e2e-mt-31899000-serving-cert" not found Jul 10 00:59:52.862331 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.862312 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051eba95-7161-4990-9954-0c80f59c8eea-openshift-service-ca\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:52.869701 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:52.869682 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgq57\" (UniqueName: \"kubernetes.io/projected/051eba95-7161-4990-9954-0c80f59c8eea-kube-api-access-jgq57\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:53.365962 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:53.365920 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:53.368490 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:53.368470 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls\") pod \"maas-api-e2e-mt-31899000-9d5fd6585-dqptk\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:53.540346 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:53.540313 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:53.671271 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:53.671224 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk"] Jul 10 00:59:53.673916 ip-10-0-131-148 kubenswrapper[2579]: W0710 00:59:53.673881 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod051eba95_7161_4990_9954_0c80f59c8eea.slice/crio-bb7271accf85d4795d59f29a6cf9049f815eec1515937856eaaae85850fb50ba WatchSource:0}: Error finding container bb7271accf85d4795d59f29a6cf9049f815eec1515937856eaaae85850fb50ba: Status 404 returned error can't find the container with id bb7271accf85d4795d59f29a6cf9049f815eec1515937856eaaae85850fb50ba Jul 10 00:59:54.520283 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:54.520173 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" event={"ID":"051eba95-7161-4990-9954-0c80f59c8eea","Type":"ContainerStarted","Data":"5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f"} Jul 10 00:59:54.520283 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:54.520228 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" event={"ID":"051eba95-7161-4990-9954-0c80f59c8eea","Type":"ContainerStarted","Data":"bb7271accf85d4795d59f29a6cf9049f815eec1515937856eaaae85850fb50ba"} Jul 10 00:59:54.520460 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:54.520285 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 00:59:54.536703 ip-10-0-131-148 kubenswrapper[2579]: I0710 00:59:54.536660 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" podStartSLOduration=2.115202832 podStartE2EDuration="2.536647817s" totalImagesPullingTime="421.444985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 00:59:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 00:59:54.534509345 +0000 UTC m=+2014.977238396" watchObservedRunningTime="2026-07-10 00:59:54.536647817 +0000 UTC m=+2014.979376924" Jul 10 01:00:00.530442 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:00.530416 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 01:00:35.187600 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.187567 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk"] Jul 10 01:00:35.188152 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.187777 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" podUID="051eba95-7161-4990-9954-0c80f59c8eea" containerName="maas-api" containerID="cri-o://5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f" gracePeriod=30 Jul 10 01:00:35.428614 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.428585 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 01:00:35.507893 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.507817 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls\") pod \"051eba95-7161-4990-9954-0c80f59c8eea\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " Jul 10 01:00:35.507893 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.507877 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051eba95-7161-4990-9954-0c80f59c8eea-openshift-service-ca\") pod \"051eba95-7161-4990-9954-0c80f59c8eea\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " Jul 10 01:00:35.508055 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.507904 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgq57\" (UniqueName: \"kubernetes.io/projected/051eba95-7161-4990-9954-0c80f59c8eea-kube-api-access-jgq57\") pod \"051eba95-7161-4990-9954-0c80f59c8eea\" (UID: \"051eba95-7161-4990-9954-0c80f59c8eea\") " Jul 10 01:00:35.508225 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.508201 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/051eba95-7161-4990-9954-0c80f59c8eea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "051eba95-7161-4990-9954-0c80f59c8eea" (UID: "051eba95-7161-4990-9954-0c80f59c8eea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:00:35.510022 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.509997 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "051eba95-7161-4990-9954-0c80f59c8eea" (UID: "051eba95-7161-4990-9954-0c80f59c8eea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:00:35.510167 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.510110 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/051eba95-7161-4990-9954-0c80f59c8eea-kube-api-access-jgq57" (OuterVolumeSpecName: "kube-api-access-jgq57") pod "051eba95-7161-4990-9954-0c80f59c8eea" (UID: "051eba95-7161-4990-9954-0c80f59c8eea"). InnerVolumeSpecName "kube-api-access-jgq57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:00:35.609001 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.608966 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/051eba95-7161-4990-9954-0c80f59c8eea-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:00:35.609001 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.608994 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgq57\" (UniqueName: \"kubernetes.io/projected/051eba95-7161-4990-9954-0c80f59c8eea-kube-api-access-jgq57\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:00:35.609001 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.609006 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/051eba95-7161-4990-9954-0c80f59c8eea-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:00:35.683693 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.683659 2579 generic.go:356] "Generic (PLEG): container finished" podID="051eba95-7161-4990-9954-0c80f59c8eea" containerID="5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f" exitCode=0 Jul 10 01:00:35.683856 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.683707 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" event={"ID":"051eba95-7161-4990-9954-0c80f59c8eea","Type":"ContainerDied","Data":"5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f"} Jul 10 01:00:35.683856 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.683721 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" Jul 10 01:00:35.683856 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.683794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk" event={"ID":"051eba95-7161-4990-9954-0c80f59c8eea","Type":"ContainerDied","Data":"bb7271accf85d4795d59f29a6cf9049f815eec1515937856eaaae85850fb50ba"} Jul 10 01:00:35.683856 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.683825 2579 scope.go:117] "RemoveContainer" containerID="5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f" Jul 10 01:00:35.692441 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.692426 2579 scope.go:117] "RemoveContainer" containerID="5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f" Jul 10 01:00:35.692649 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:00:35.692631 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f\": container with ID starting with 5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f not found: ID does not exist" containerID="5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f" Jul 10 01:00:35.692708 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.692656 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f"} err="failed to get container status \"5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f\": rpc error: code = NotFound desc = could not find container \"5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f\": container with ID starting with 5cc610d326e7dfd13096fe355a50c4b4d85ee7fcf1ddd42bf88c33a46c39163f not found: ID does not exist" Jul 10 01:00:35.705853 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.705824 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk"] Jul 10 01:00:35.709617 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:35.709591 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31899000-9d5fd6585-dqptk"] Jul 10 01:00:36.025637 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:00:36.025604 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="051eba95-7161-4990-9954-0c80f59c8eea" path="/var/lib/kubelet/pods/051eba95-7161-4990-9954-0c80f59c8eea/volumes" Jul 10 01:01:20.051496 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:01:20.051466 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 01:01:20.056881 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:01:20.056857 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 01:02:26.485184 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.485152 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t"] Jul 10 01:02:26.485731 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.485447 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="051eba95-7161-4990-9954-0c80f59c8eea" containerName="maas-api" Jul 10 01:02:26.485731 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.485459 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="051eba95-7161-4990-9954-0c80f59c8eea" containerName="maas-api" Jul 10 01:02:26.485731 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.485512 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="051eba95-7161-4990-9954-0c80f59c8eea" containerName="maas-api" Jul 10 01:02:26.488295 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.488275 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.490312 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.490293 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-85dfb6-serving-cert\"" Jul 10 01:02:26.498404 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.498383 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t"] Jul 10 01:02:26.611593 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.611558 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd0c991-d1ff-43ee-b469-13d356d4888b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.611759 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.611610 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slzv5\" (UniqueName: \"kubernetes.io/projected/3dd0c991-d1ff-43ee-b469-13d356d4888b-kube-api-access-slzv5\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.611759 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.611685 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dd0c991-d1ff-43ee-b469-13d356d4888b-maas-api-tls\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.712716 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.712673 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dd0c991-d1ff-43ee-b469-13d356d4888b-maas-api-tls\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.712894 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.712742 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd0c991-d1ff-43ee-b469-13d356d4888b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.712894 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.712784 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slzv5\" (UniqueName: \"kubernetes.io/projected/3dd0c991-d1ff-43ee-b469-13d356d4888b-kube-api-access-slzv5\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.713359 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.713330 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd0c991-d1ff-43ee-b469-13d356d4888b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.715452 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.715428 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dd0c991-d1ff-43ee-b469-13d356d4888b-maas-api-tls\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.720439 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.720414 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slzv5\" (UniqueName: \"kubernetes.io/projected/3dd0c991-d1ff-43ee-b469-13d356d4888b-kube-api-access-slzv5\") pod \"maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.799061 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.798980 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:26.928904 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.928876 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t"] Jul 10 01:02:26.932153 ip-10-0-131-148 kubenswrapper[2579]: W0710 01:02:26.932125 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd0c991_d1ff_43ee_b469_13d356d4888b.slice/crio-bee2c9757691638bd29bf507eaf9ecf571daf2a981474e92d53b00fcd956c304 WatchSource:0}: Error finding container bee2c9757691638bd29bf507eaf9ecf571daf2a981474e92d53b00fcd956c304: Status 404 returned error can't find the container with id bee2c9757691638bd29bf507eaf9ecf571daf2a981474e92d53b00fcd956c304 Jul 10 01:02:26.933423 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:26.933402 2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 01:02:27.129134 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:27.129095 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" event={"ID":"3dd0c991-d1ff-43ee-b469-13d356d4888b","Type":"ContainerStarted","Data":"bee2c9757691638bd29bf507eaf9ecf571daf2a981474e92d53b00fcd956c304"} Jul 10 01:02:28.134012 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:28.133972 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" event={"ID":"3dd0c991-d1ff-43ee-b469-13d356d4888b","Type":"ContainerStarted","Data":"8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe"} Jul 10 01:02:28.134430 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:28.134096 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:28.150861 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:28.150813 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" podStartSLOduration=1.657084228 podStartE2EDuration="2.15079708s" totalImagesPullingTime="493.712852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:02:28.149893589 +0000 UTC m=+2168.592622645" watchObservedRunningTime="2026-07-10 01:02:28.15079708 +0000 UTC m=+2168.593526130" Jul 10 01:02:34.143473 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.143439 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:02:34.394360 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.394275 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq"] Jul 10 01:02:34.397656 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.397639 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.400021 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.400001 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-8f8f47-serving-cert\"" Jul 10 01:02:34.412163 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.412141 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq"] Jul 10 01:02:34.482448 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.482415 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d70235-6694-465c-929d-28db1dd73ab5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.482610 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.482473 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xt5p\" (UniqueName: \"kubernetes.io/projected/f3d70235-6694-465c-929d-28db1dd73ab5-kube-api-access-2xt5p\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.482610 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.482500 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.583322 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.583292 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xt5p\" (UniqueName: \"kubernetes.io/projected/f3d70235-6694-465c-929d-28db1dd73ab5-kube-api-access-2xt5p\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.583496 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.583345 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.583496 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.583396 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d70235-6694-465c-929d-28db1dd73ab5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.583587 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:02:34.583517 2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-serving-cert: secret "maas-api-e2e-inf-b-8f8f47-serving-cert" not found Jul 10 01:02:34.583587 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:02:34.583583 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls podName:f3d70235-6694-465c-929d-28db1dd73ab5 nodeName:}" failed. No retries permitted until 2026-07-10 01:02:35.083566199 +0000 UTC m=+2175.526295228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls") pod "maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" (UID: "f3d70235-6694-465c-929d-28db1dd73ab5") : secret "maas-api-e2e-inf-b-8f8f47-serving-cert" not found Jul 10 01:02:34.584012 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.583994 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d70235-6694-465c-929d-28db1dd73ab5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:34.594294 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:34.594270 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xt5p\" (UniqueName: \"kubernetes.io/projected/f3d70235-6694-465c-929d-28db1dd73ab5-kube-api-access-2xt5p\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:35.087087 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:35.087045 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:35.089594 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:35.089574 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:35.308503 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:35.308468 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:35.443229 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:35.443035 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq"] Jul 10 01:02:35.445970 ip-10-0-131-148 kubenswrapper[2579]: W0710 01:02:35.445944 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d70235_6694_465c_929d_28db1dd73ab5.slice/crio-a83955bb17d5fc6ca6b6b61126e68f867e9c5f0419254450eae5a48eb6828a53 WatchSource:0}: Error finding container a83955bb17d5fc6ca6b6b61126e68f867e9c5f0419254450eae5a48eb6828a53: Status 404 returned error can't find the container with id a83955bb17d5fc6ca6b6b61126e68f867e9c5f0419254450eae5a48eb6828a53 Jul 10 01:02:36.167091 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:36.167056 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" event={"ID":"f3d70235-6694-465c-929d-28db1dd73ab5","Type":"ContainerStarted","Data":"b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d"} Jul 10 01:02:36.167274 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:36.167107 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" event={"ID":"f3d70235-6694-465c-929d-28db1dd73ab5","Type":"ContainerStarted","Data":"a83955bb17d5fc6ca6b6b61126e68f867e9c5f0419254450eae5a48eb6828a53"} Jul 10 01:02:36.167274 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:36.167152 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:02:36.181782 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:36.181719 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" podStartSLOduration=1.8367946800000001 podStartE2EDuration="2.181704197s" totalImagesPullingTime="344.909517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:02:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:02:36.180998693 +0000 UTC m=+2176.623727747" watchObservedRunningTime="2026-07-10 01:02:36.181704197 +0000 UTC m=+2176.624433248" Jul 10 01:02:42.177106 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:02:42.177080 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:03:28.796459 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:28.796432 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t"] Jul 10 01:03:28.796949 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:28.796661 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" podUID="3dd0c991-d1ff-43ee-b469-13d356d4888b" containerName="maas-api" containerID="cri-o://8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe" gracePeriod=30 Jul 10 01:03:29.138625 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.138587 2579 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" podUID="3dd0c991-d1ff-43ee-b469-13d356d4888b" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.29:8443/health\": dial tcp 10.132.0.29:8443: connect: connection refused" Jul 10 01:03:29.233243 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.233217 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:03:29.238160 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.238139 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dd0c991-d1ff-43ee-b469-13d356d4888b-maas-api-tls\") pod \"3dd0c991-d1ff-43ee-b469-13d356d4888b\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " Jul 10 01:03:29.238318 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.238233 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd0c991-d1ff-43ee-b469-13d356d4888b-openshift-service-ca\") pod \"3dd0c991-d1ff-43ee-b469-13d356d4888b\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " Jul 10 01:03:29.238318 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.238307 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slzv5\" (UniqueName: \"kubernetes.io/projected/3dd0c991-d1ff-43ee-b469-13d356d4888b-kube-api-access-slzv5\") pod \"3dd0c991-d1ff-43ee-b469-13d356d4888b\" (UID: \"3dd0c991-d1ff-43ee-b469-13d356d4888b\") " Jul 10 01:03:29.238603 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.238575 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dd0c991-d1ff-43ee-b469-13d356d4888b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3dd0c991-d1ff-43ee-b469-13d356d4888b" (UID: "3dd0c991-d1ff-43ee-b469-13d356d4888b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:03:29.240189 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.240162 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3dd0c991-d1ff-43ee-b469-13d356d4888b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3dd0c991-d1ff-43ee-b469-13d356d4888b" (UID: "3dd0c991-d1ff-43ee-b469-13d356d4888b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:03:29.240301 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.240280 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dd0c991-d1ff-43ee-b469-13d356d4888b-kube-api-access-slzv5" (OuterVolumeSpecName: "kube-api-access-slzv5") pod "3dd0c991-d1ff-43ee-b469-13d356d4888b" (UID: "3dd0c991-d1ff-43ee-b469-13d356d4888b"). InnerVolumeSpecName "kube-api-access-slzv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:03:29.338892 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.338804 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3dd0c991-d1ff-43ee-b469-13d356d4888b-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:03:29.338892 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.338836 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slzv5\" (UniqueName: \"kubernetes.io/projected/3dd0c991-d1ff-43ee-b469-13d356d4888b-kube-api-access-slzv5\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:03:29.338892 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.338849 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3dd0c991-d1ff-43ee-b469-13d356d4888b-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:03:29.385170 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.385142 2579 generic.go:356] "Generic (PLEG): container finished" podID="3dd0c991-d1ff-43ee-b469-13d356d4888b" containerID="8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe" exitCode=0 Jul 10 01:03:29.385340 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.385189 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" Jul 10 01:03:29.385340 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.385213 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" event={"ID":"3dd0c991-d1ff-43ee-b469-13d356d4888b","Type":"ContainerDied","Data":"8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe"} Jul 10 01:03:29.385340 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.385277 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t" event={"ID":"3dd0c991-d1ff-43ee-b469-13d356d4888b","Type":"ContainerDied","Data":"bee2c9757691638bd29bf507eaf9ecf571daf2a981474e92d53b00fcd956c304"} Jul 10 01:03:29.385340 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.385310 2579 scope.go:117] "RemoveContainer" containerID="8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe" Jul 10 01:03:29.393992 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.393972 2579 scope.go:117] "RemoveContainer" containerID="8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe" Jul 10 01:03:29.394263 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:03:29.394231 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe\": container with ID starting with 8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe not found: ID does not exist" containerID="8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe" Jul 10 01:03:29.394321 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.394289 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe"} err="failed to get container status \"8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe\": rpc error: code = NotFound desc = could not find container \"8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe\": container with ID starting with 8cb93a332bb57e9e269bdbb5c4fc459bc755745fcff0b4a717199cac0784b5fe not found: ID does not exist" Jul 10 01:03:29.404650 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.404631 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t"] Jul 10 01:03:29.410745 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:29.410725 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-85dfb6-86d87cdccd-88f7t"] Jul 10 01:03:30.026623 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:03:30.026592 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dd0c991-d1ff-43ee-b469-13d356d4888b" path="/var/lib/kubelet/pods/3dd0c991-d1ff-43ee-b469-13d356d4888b/volumes" Jul 10 01:04:11.694681 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:11.694597 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq"] Jul 10 01:04:11.695240 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:11.694819 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" podUID="f3d70235-6694-465c-929d-28db1dd73ab5" containerName="maas-api" containerID="cri-o://b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d" gracePeriod=30 Jul 10 01:04:12.134777 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.134752 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:04:12.296585 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.296487 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d70235-6694-465c-929d-28db1dd73ab5-openshift-service-ca\") pod \"f3d70235-6694-465c-929d-28db1dd73ab5\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " Jul 10 01:04:12.296585 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.296527 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xt5p\" (UniqueName: \"kubernetes.io/projected/f3d70235-6694-465c-929d-28db1dd73ab5-kube-api-access-2xt5p\") pod \"f3d70235-6694-465c-929d-28db1dd73ab5\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " Jul 10 01:04:12.296585 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.296576 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls\") pod \"f3d70235-6694-465c-929d-28db1dd73ab5\" (UID: \"f3d70235-6694-465c-929d-28db1dd73ab5\") " Jul 10 01:04:12.296875 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.296852 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d70235-6694-465c-929d-28db1dd73ab5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3d70235-6694-465c-929d-28db1dd73ab5" (UID: "f3d70235-6694-465c-929d-28db1dd73ab5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:04:12.298734 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.298709 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3d70235-6694-465c-929d-28db1dd73ab5" (UID: "f3d70235-6694-465c-929d-28db1dd73ab5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:04:12.298816 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.298743 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d70235-6694-465c-929d-28db1dd73ab5-kube-api-access-2xt5p" (OuterVolumeSpecName: "kube-api-access-2xt5p") pod "f3d70235-6694-465c-929d-28db1dd73ab5" (UID: "f3d70235-6694-465c-929d-28db1dd73ab5"). InnerVolumeSpecName "kube-api-access-2xt5p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:04:12.398124 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.398094 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3d70235-6694-465c-929d-28db1dd73ab5-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:12.398124 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.398118 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xt5p\" (UniqueName: \"kubernetes.io/projected/f3d70235-6694-465c-929d-28db1dd73ab5-kube-api-access-2xt5p\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:12.398124 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.398128 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3d70235-6694-465c-929d-28db1dd73ab5-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:12.558763 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.558679 2579 generic.go:356] "Generic (PLEG): container finished" podID="f3d70235-6694-465c-929d-28db1dd73ab5" containerID="b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d" exitCode=0 Jul 10 01:04:12.558763 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.558725 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" Jul 10 01:04:12.558941 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.558767 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" event={"ID":"f3d70235-6694-465c-929d-28db1dd73ab5","Type":"ContainerDied","Data":"b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d"} Jul 10 01:04:12.558941 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.558812 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq" event={"ID":"f3d70235-6694-465c-929d-28db1dd73ab5","Type":"ContainerDied","Data":"a83955bb17d5fc6ca6b6b61126e68f867e9c5f0419254450eae5a48eb6828a53"} Jul 10 01:04:12.558941 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.558832 2579 scope.go:117] "RemoveContainer" containerID="b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d" Jul 10 01:04:12.567554 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.567530 2579 scope.go:117] "RemoveContainer" containerID="b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d" Jul 10 01:04:12.567796 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:04:12.567778 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d\": container with ID starting with b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d not found: ID does not exist" containerID="b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d" Jul 10 01:04:12.567845 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.567810 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d"} err="failed to get container status \"b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d\": rpc error: code = NotFound desc = could not find container \"b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d\": container with ID starting with b07c4f162ef52ca28e7e4847dd6672693bf27452814e6b2db1a4ccacaf52d50d not found: ID does not exist" Jul 10 01:04:12.578976 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.578956 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq"] Jul 10 01:04:12.583487 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:12.583461 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f8f47-547bfb8b5f-4q9nq"] Jul 10 01:04:14.026239 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:14.026203 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d70235-6694-465c-929d-28db1dd73ab5" path="/var/lib/kubelet/pods/f3d70235-6694-465c-929d-28db1dd73ab5/volumes" Jul 10 01:04:21.015294 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.015239 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc"] Jul 10 01:04:21.015802 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.015674 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dd0c991-d1ff-43ee-b469-13d356d4888b" containerName="maas-api" Jul 10 01:04:21.015802 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.015693 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dd0c991-d1ff-43ee-b469-13d356d4888b" containerName="maas-api" Jul 10 01:04:21.015802 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.015704 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d70235-6694-465c-929d-28db1dd73ab5" containerName="maas-api" Jul 10 01:04:21.015802 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.015711 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d70235-6694-465c-929d-28db1dd73ab5" containerName="maas-api" Jul 10 01:04:21.015802 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.015792 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dd0c991-d1ff-43ee-b469-13d356d4888b" containerName="maas-api" Jul 10 01:04:21.015802 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.015802 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d70235-6694-465c-929d-28db1dd73ab5" containerName="maas-api" Jul 10 01:04:21.026629 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.026594 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.029286 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.029229 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-7e8785-serving-cert\"" Jul 10 01:04:21.029449 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.029319 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc"] Jul 10 01:04:21.079318 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.079280 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpcpd\" (UniqueName: \"kubernetes.io/projected/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-kube-api-access-lpcpd\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.079508 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.079357 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-maas-api-tls\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.079508 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.079378 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-openshift-service-ca\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.179878 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.179838 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpcpd\" (UniqueName: \"kubernetes.io/projected/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-kube-api-access-lpcpd\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.179878 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.179884 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-maas-api-tls\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.180103 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.180003 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-openshift-service-ca\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.180624 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.180556 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-openshift-service-ca\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.182429 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.182408 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-maas-api-tls\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.193660 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.193632 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpcpd\" (UniqueName: \"kubernetes.io/projected/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-kube-api-access-lpcpd\") pod \"maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.337970 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.337932 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:21.476703 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.476671 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc"] Jul 10 01:04:21.479948 ip-10-0-131-148 kubenswrapper[2579]: W0710 01:04:21.479921 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b57fa7_df88_4672_bff9_17ce4f7d0b1f.slice/crio-b5f25027866d7fcd2208910a68e80df2ff18b03c929ac4bf099dc8d9f9f55a7d WatchSource:0}: Error finding container b5f25027866d7fcd2208910a68e80df2ff18b03c929ac4bf099dc8d9f9f55a7d: Status 404 returned error can't find the container with id b5f25027866d7fcd2208910a68e80df2ff18b03c929ac4bf099dc8d9f9f55a7d Jul 10 01:04:21.595276 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:21.595179 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" event={"ID":"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f","Type":"ContainerStarted","Data":"b5f25027866d7fcd2208910a68e80df2ff18b03c929ac4bf099dc8d9f9f55a7d"} Jul 10 01:04:22.600995 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:22.600960 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" event={"ID":"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f","Type":"ContainerStarted","Data":"bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52"} Jul 10 01:04:22.601405 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:22.601093 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:22.617532 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:22.617485 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" podStartSLOduration=2.134584711 podStartE2EDuration="2.617469764s" totalImagesPullingTime="482.885053ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:04:22.615548803 +0000 UTC m=+2283.058277855" watchObservedRunningTime="2026-07-10 01:04:22.617469764 +0000 UTC m=+2283.060198815" Jul 10 01:04:28.610667 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:28.610640 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:29.086286 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.086237 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv"] Jul 10 01:04:29.089496 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.089473 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.091638 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.091616 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0025fe-serving-cert\"" Jul 10 01:04:29.099152 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.099126 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv"] Jul 10 01:04:29.137061 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.137029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca9951a8-445d-4022-b827-dfdc5c775eb8-maas-api-tls\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.137234 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.137069 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qnnr\" (UniqueName: \"kubernetes.io/projected/ca9951a8-445d-4022-b827-dfdc5c775eb8-kube-api-access-9qnnr\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.137234 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.137161 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9951a8-445d-4022-b827-dfdc5c775eb8-openshift-service-ca\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.237663 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.237623 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qnnr\" (UniqueName: \"kubernetes.io/projected/ca9951a8-445d-4022-b827-dfdc5c775eb8-kube-api-access-9qnnr\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.237839 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.237686 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9951a8-445d-4022-b827-dfdc5c775eb8-openshift-service-ca\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.237839 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.237733 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca9951a8-445d-4022-b827-dfdc5c775eb8-maas-api-tls\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.238465 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.238367 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9951a8-445d-4022-b827-dfdc5c775eb8-openshift-service-ca\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.240219 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.240197 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca9951a8-445d-4022-b827-dfdc5c775eb8-maas-api-tls\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.245458 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.245433 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qnnr\" (UniqueName: \"kubernetes.io/projected/ca9951a8-445d-4022-b827-dfdc5c775eb8-kube-api-access-9qnnr\") pod \"maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.400406 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.400295 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:29.538985 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.538867 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv"] Jul 10 01:04:29.541670 ip-10-0-131-148 kubenswrapper[2579]: W0710 01:04:29.541641 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9951a8_445d_4022_b827_dfdc5c775eb8.slice/crio-3f1d9ae8b1bdd748ca021794c7e982cc9ee4eb48c68dcc2f010986aee6219e8d WatchSource:0}: Error finding container 3f1d9ae8b1bdd748ca021794c7e982cc9ee4eb48c68dcc2f010986aee6219e8d: Status 404 returned error can't find the container with id 3f1d9ae8b1bdd748ca021794c7e982cc9ee4eb48c68dcc2f010986aee6219e8d Jul 10 01:04:29.629611 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:29.629567 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" event={"ID":"ca9951a8-445d-4022-b827-dfdc5c775eb8","Type":"ContainerStarted","Data":"3f1d9ae8b1bdd748ca021794c7e982cc9ee4eb48c68dcc2f010986aee6219e8d"} Jul 10 01:04:30.635038 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:30.634998 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" event={"ID":"ca9951a8-445d-4022-b827-dfdc5c775eb8","Type":"ContainerStarted","Data":"4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268"} Jul 10 01:04:30.635451 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:30.635135 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:30.653947 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:30.653897 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" podStartSLOduration=1.278055892 podStartE2EDuration="1.653880447s" totalImagesPullingTime="375.824555ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:04:30.651589129 +0000 UTC m=+2291.094318178" watchObservedRunningTime="2026-07-10 01:04:30.653880447 +0000 UTC m=+2291.096609497" Jul 10 01:04:36.646237 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:36.646211 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:42.372483 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.372449 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc"] Jul 10 01:04:42.372969 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.372671 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" podUID="b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" containerName="maas-api" containerID="cri-o://bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52" gracePeriod=30 Jul 10 01:04:42.609029 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.609007 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:42.645198 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.645120 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpcpd\" (UniqueName: \"kubernetes.io/projected/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-kube-api-access-lpcpd\") pod \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " Jul 10 01:04:42.645198 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.645168 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-maas-api-tls\") pod \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " Jul 10 01:04:42.645198 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.645195 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-openshift-service-ca\") pod \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\" (UID: \"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f\") " Jul 10 01:04:42.645526 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.645502 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" (UID: "b1b57fa7-df88-4672-bff9-17ce4f7d0b1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:04:42.647315 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.647290 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" (UID: "b1b57fa7-df88-4672-bff9-17ce4f7d0b1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:04:42.647665 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.647642 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-kube-api-access-lpcpd" (OuterVolumeSpecName: "kube-api-access-lpcpd") pod "b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" (UID: "b1b57fa7-df88-4672-bff9-17ce4f7d0b1f"). InnerVolumeSpecName "kube-api-access-lpcpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:04:42.685518 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.685481 2579 generic.go:356] "Generic (PLEG): container finished" podID="b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" containerID="bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52" exitCode=0 Jul 10 01:04:42.685654 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.685530 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" Jul 10 01:04:42.685654 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.685529 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" event={"ID":"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f","Type":"ContainerDied","Data":"bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52"} Jul 10 01:04:42.685654 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.685638 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc" event={"ID":"b1b57fa7-df88-4672-bff9-17ce4f7d0b1f","Type":"ContainerDied","Data":"b5f25027866d7fcd2208910a68e80df2ff18b03c929ac4bf099dc8d9f9f55a7d"} Jul 10 01:04:42.685755 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.685666 2579 scope.go:117] "RemoveContainer" containerID="bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52" Jul 10 01:04:42.694227 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.694211 2579 scope.go:117] "RemoveContainer" containerID="bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52" Jul 10 01:04:42.694481 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:04:42.694463 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52\": container with ID starting with bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52 not found: ID does not exist" containerID="bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52" Jul 10 01:04:42.694541 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.694491 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52"} err="failed to get container status \"bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52\": rpc error: code = NotFound desc = could not find container \"bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52\": container with ID starting with bfdafe598874db6594980bbd08cc2a7401e4c898f2f065c842603c39dd833e52 not found: ID does not exist" Jul 10 01:04:42.704921 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.704899 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc"] Jul 10 01:04:42.708292 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.708269 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7e8785-6b459f4fd4-9d4hc"] Jul 10 01:04:42.746049 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.746014 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpcpd\" (UniqueName: \"kubernetes.io/projected/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-kube-api-access-lpcpd\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:42.746049 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.746044 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:42.746049 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:42.746054 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:44.025943 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:44.025898 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" path="/var/lib/kubelet/pods/b1b57fa7-df88-4672-bff9-17ce4f7d0b1f/volumes" Jul 10 01:04:49.696042 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:49.696008 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv"] Jul 10 01:04:49.696675 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:49.696317 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" podUID="ca9951a8-445d-4022-b827-dfdc5c775eb8" containerName="maas-api" containerID="cri-o://4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268" gracePeriod=30 Jul 10 01:04:49.941748 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:49.941725 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:50.010086 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.010000 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qnnr\" (UniqueName: \"kubernetes.io/projected/ca9951a8-445d-4022-b827-dfdc5c775eb8-kube-api-access-9qnnr\") pod \"ca9951a8-445d-4022-b827-dfdc5c775eb8\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " Jul 10 01:04:50.010286 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.010090 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9951a8-445d-4022-b827-dfdc5c775eb8-openshift-service-ca\") pod \"ca9951a8-445d-4022-b827-dfdc5c775eb8\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " Jul 10 01:04:50.010286 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.010120 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca9951a8-445d-4022-b827-dfdc5c775eb8-maas-api-tls\") pod \"ca9951a8-445d-4022-b827-dfdc5c775eb8\" (UID: \"ca9951a8-445d-4022-b827-dfdc5c775eb8\") " Jul 10 01:04:50.010519 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.010493 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca9951a8-445d-4022-b827-dfdc5c775eb8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca9951a8-445d-4022-b827-dfdc5c775eb8" (UID: "ca9951a8-445d-4022-b827-dfdc5c775eb8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:04:50.012223 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.012198 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca9951a8-445d-4022-b827-dfdc5c775eb8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca9951a8-445d-4022-b827-dfdc5c775eb8" (UID: "ca9951a8-445d-4022-b827-dfdc5c775eb8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:04:50.012336 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.012304 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca9951a8-445d-4022-b827-dfdc5c775eb8-kube-api-access-9qnnr" (OuterVolumeSpecName: "kube-api-access-9qnnr") pod "ca9951a8-445d-4022-b827-dfdc5c775eb8" (UID: "ca9951a8-445d-4022-b827-dfdc5c775eb8"). InnerVolumeSpecName "kube-api-access-9qnnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:04:50.110969 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.110926 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qnnr\" (UniqueName: \"kubernetes.io/projected/ca9951a8-445d-4022-b827-dfdc5c775eb8-kube-api-access-9qnnr\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:50.110969 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.110961 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca9951a8-445d-4022-b827-dfdc5c775eb8-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:50.110969 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.110971 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca9951a8-445d-4022-b827-dfdc5c775eb8-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:04:50.720757 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.720720 2579 generic.go:356] "Generic (PLEG): container finished" podID="ca9951a8-445d-4022-b827-dfdc5c775eb8" containerID="4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268" exitCode=0 Jul 10 01:04:50.721170 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.720766 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" Jul 10 01:04:50.721170 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.720808 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" event={"ID":"ca9951a8-445d-4022-b827-dfdc5c775eb8","Type":"ContainerDied","Data":"4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268"} Jul 10 01:04:50.721170 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.720860 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv" event={"ID":"ca9951a8-445d-4022-b827-dfdc5c775eb8","Type":"ContainerDied","Data":"3f1d9ae8b1bdd748ca021794c7e982cc9ee4eb48c68dcc2f010986aee6219e8d"} Jul 10 01:04:50.721170 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.720881 2579 scope.go:117] "RemoveContainer" containerID="4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268" Jul 10 01:04:50.729041 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.729016 2579 scope.go:117] "RemoveContainer" containerID="4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268" Jul 10 01:04:50.729378 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:04:50.729361 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268\": container with ID starting with 4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268 not found: ID does not exist" containerID="4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268" Jul 10 01:04:50.729474 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.729387 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268"} err="failed to get container status \"4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268\": rpc error: code = NotFound desc = could not find container \"4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268\": container with ID starting with 4ada17dd0881d1774fb93f12af56e264deb39ef0ee20376131b226f8e7843268 not found: ID does not exist" Jul 10 01:04:50.735066 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.735041 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv"] Jul 10 01:04:50.737038 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:50.737015 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0025fe-6d4cf9bfff-lr2sv"] Jul 10 01:04:52.026280 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:52.026234 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca9951a8-445d-4022-b827-dfdc5c775eb8" path="/var/lib/kubelet/pods/ca9951a8-445d-4022-b827-dfdc5c775eb8/volumes" Jul 10 01:04:59.335212 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.335166 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n"] Jul 10 01:04:59.335715 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.335648 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" containerName="maas-api" Jul 10 01:04:59.335715 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.335666 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" containerName="maas-api" Jul 10 01:04:59.335715 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.335695 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca9951a8-445d-4022-b827-dfdc5c775eb8" containerName="maas-api" Jul 10 01:04:59.335715 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.335703 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca9951a8-445d-4022-b827-dfdc5c775eb8" containerName="maas-api" Jul 10 01:04:59.335923 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.335776 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1b57fa7-df88-4672-bff9-17ce4f7d0b1f" containerName="maas-api" Jul 10 01:04:59.335923 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.335789 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca9951a8-445d-4022-b827-dfdc5c775eb8" containerName="maas-api" Jul 10 01:04:59.338661 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.338639 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.340783 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.340761 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-48a750-serving-cert\"" Jul 10 01:04:59.347779 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.347756 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n"] Jul 10 01:04:59.492920 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.492880 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b583dba-3f9b-4ed9-a08c-b138f7005fed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.493115 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.492932 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtph2\" (UniqueName: \"kubernetes.io/projected/7b583dba-3f9b-4ed9-a08c-b138f7005fed-kube-api-access-mtph2\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.493115 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.493029 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b583dba-3f9b-4ed9-a08c-b138f7005fed-maas-api-tls\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.594097 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.593993 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtph2\" (UniqueName: \"kubernetes.io/projected/7b583dba-3f9b-4ed9-a08c-b138f7005fed-kube-api-access-mtph2\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.594097 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.594052 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b583dba-3f9b-4ed9-a08c-b138f7005fed-maas-api-tls\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.594387 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.594120 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b583dba-3f9b-4ed9-a08c-b138f7005fed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.594805 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.594781 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b583dba-3f9b-4ed9-a08c-b138f7005fed-openshift-service-ca\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.596682 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.596663 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b583dba-3f9b-4ed9-a08c-b138f7005fed-maas-api-tls\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.602385 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.602355 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtph2\" (UniqueName: \"kubernetes.io/projected/7b583dba-3f9b-4ed9-a08c-b138f7005fed-kube-api-access-mtph2\") pod \"maas-api-e2e-shared-a-48a750-78c4684846-s4g7n\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.649926 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.649885 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:04:59.788461 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:04:59.788430 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n"] Jul 10 01:04:59.792672 ip-10-0-131-148 kubenswrapper[2579]: W0710 01:04:59.792645 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b583dba_3f9b_4ed9_a08c_b138f7005fed.slice/crio-29305aa594fb62c8c4ee26b9013670984b1f308a3c7246714accce893ac29d8c WatchSource:0}: Error finding container 29305aa594fb62c8c4ee26b9013670984b1f308a3c7246714accce893ac29d8c: Status 404 returned error can't find the container with id 29305aa594fb62c8c4ee26b9013670984b1f308a3c7246714accce893ac29d8c Jul 10 01:05:00.766659 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:00.766624 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" event={"ID":"7b583dba-3f9b-4ed9-a08c-b138f7005fed","Type":"ContainerStarted","Data":"b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da"} Jul 10 01:05:00.767088 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:00.766683 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" event={"ID":"7b583dba-3f9b-4ed9-a08c-b138f7005fed","Type":"ContainerStarted","Data":"29305aa594fb62c8c4ee26b9013670984b1f308a3c7246714accce893ac29d8c"} Jul 10 01:05:00.767088 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:00.766718 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:05:00.784216 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:00.784162 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" podStartSLOduration=1.467229016 podStartE2EDuration="1.784146872s" totalImagesPullingTime="316.917856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:04:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:05:00.782654076 +0000 UTC m=+2321.225383142" watchObservedRunningTime="2026-07-10 01:05:00.784146872 +0000 UTC m=+2321.226875949" Jul 10 01:05:06.777727 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:06.777696 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:05:07.473396 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.473358 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d"] Jul 10 01:05:07.477599 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.477568 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.479739 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.479714 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-aaea96-serving-cert\"" Jul 10 01:05:07.487945 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.487915 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d"] Jul 10 01:05:07.565402 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.565360 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh5l2\" (UniqueName: \"kubernetes.io/projected/3407cca8-9d7f-4414-9b26-2ed27b54432f-kube-api-access-xh5l2\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.565580 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.565433 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3407cca8-9d7f-4414-9b26-2ed27b54432f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.565580 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.565474 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.666545 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.666505 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3407cca8-9d7f-4414-9b26-2ed27b54432f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.666749 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.666558 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.666749 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.666629 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh5l2\" (UniqueName: \"kubernetes.io/projected/3407cca8-9d7f-4414-9b26-2ed27b54432f-kube-api-access-xh5l2\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.666749 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:05:07.666741 2579 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-serving-cert: secret "maas-api-e2e-shared-b-aaea96-serving-cert" not found Jul 10 01:05:07.666902 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:05:07.666819 2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls podName:3407cca8-9d7f-4414-9b26-2ed27b54432f nodeName:}" failed. No retries permitted until 2026-07-10 01:05:08.166800847 +0000 UTC m=+2328.609529876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls") pod "maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" (UID: "3407cca8-9d7f-4414-9b26-2ed27b54432f") : secret "maas-api-e2e-shared-b-aaea96-serving-cert" not found Jul 10 01:05:07.667146 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.667125 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3407cca8-9d7f-4414-9b26-2ed27b54432f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:07.675464 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:07.675443 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh5l2\" (UniqueName: \"kubernetes.io/projected/3407cca8-9d7f-4414-9b26-2ed27b54432f-kube-api-access-xh5l2\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:08.170010 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:08.169972 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:08.172533 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:08.172509 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls\") pod \"maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:08.390223 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:08.390183 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:08.547993 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:08.547962 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d"] Jul 10 01:05:08.550669 ip-10-0-131-148 kubenswrapper[2579]: W0710 01:05:08.550635 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3407cca8_9d7f_4414_9b26_2ed27b54432f.slice/crio-886be5197312bd9896ab4c35e07fb02fec9c4eabb58d9a797eccd669736be7cf WatchSource:0}: Error finding container 886be5197312bd9896ab4c35e07fb02fec9c4eabb58d9a797eccd669736be7cf: Status 404 returned error can't find the container with id 886be5197312bd9896ab4c35e07fb02fec9c4eabb58d9a797eccd669736be7cf Jul 10 01:05:08.800550 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:08.800465 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" event={"ID":"3407cca8-9d7f-4414-9b26-2ed27b54432f","Type":"ContainerStarted","Data":"886be5197312bd9896ab4c35e07fb02fec9c4eabb58d9a797eccd669736be7cf"} Jul 10 01:05:09.806458 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:09.806429 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" event={"ID":"3407cca8-9d7f-4414-9b26-2ed27b54432f","Type":"ContainerStarted","Data":"eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2"} Jul 10 01:05:09.806862 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:09.806491 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:05:09.822360 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:09.822318 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" podStartSLOduration=2.483789188 podStartE2EDuration="2.822302736s" totalImagesPullingTime="338.513548ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 01:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:05:09.821447176 +0000 UTC m=+2330.264176221" watchObservedRunningTime="2026-07-10 01:05:09.822302736 +0000 UTC m=+2330.265031787" Jul 10 01:05:15.816072 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:05:15.815986 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:06:20.074151 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:06:20.074125 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 01:06:20.080017 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:06:20.079999 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 01:10:50.149782 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.149751 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n"] Jul 10 01:10:50.150377 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.149990 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" podUID="7b583dba-3f9b-4ed9-a08c-b138f7005fed" containerName="maas-api" containerID="cri-o://b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da" gracePeriod=30 Jul 10 01:10:50.398024 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.398000 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:10:50.454375 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.454286 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b583dba-3f9b-4ed9-a08c-b138f7005fed-openshift-service-ca\") pod \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " Jul 10 01:10:50.454375 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.454332 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtph2\" (UniqueName: \"kubernetes.io/projected/7b583dba-3f9b-4ed9-a08c-b138f7005fed-kube-api-access-mtph2\") pod \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " Jul 10 01:10:50.454608 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.454546 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b583dba-3f9b-4ed9-a08c-b138f7005fed-maas-api-tls\") pod \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\" (UID: \"7b583dba-3f9b-4ed9-a08c-b138f7005fed\") " Jul 10 01:10:50.454736 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.454712 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b583dba-3f9b-4ed9-a08c-b138f7005fed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7b583dba-3f9b-4ed9-a08c-b138f7005fed" (UID: "7b583dba-3f9b-4ed9-a08c-b138f7005fed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:10:50.454810 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.454794 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b583dba-3f9b-4ed9-a08c-b138f7005fed-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:10:50.456690 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.456662 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b583dba-3f9b-4ed9-a08c-b138f7005fed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7b583dba-3f9b-4ed9-a08c-b138f7005fed" (UID: "7b583dba-3f9b-4ed9-a08c-b138f7005fed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:10:50.456811 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.456727 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b583dba-3f9b-4ed9-a08c-b138f7005fed-kube-api-access-mtph2" (OuterVolumeSpecName: "kube-api-access-mtph2") pod "7b583dba-3f9b-4ed9-a08c-b138f7005fed" (UID: "7b583dba-3f9b-4ed9-a08c-b138f7005fed"). InnerVolumeSpecName "kube-api-access-mtph2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:10:50.555866 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.555832 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7b583dba-3f9b-4ed9-a08c-b138f7005fed-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:10:50.555866 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:50.555861 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtph2\" (UniqueName: \"kubernetes.io/projected/7b583dba-3f9b-4ed9-a08c-b138f7005fed-kube-api-access-mtph2\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:10:51.164736 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.164705 2579 generic.go:356] "Generic (PLEG): container finished" podID="7b583dba-3f9b-4ed9-a08c-b138f7005fed" containerID="b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da" exitCode=0 Jul 10 01:10:51.165207 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.164751 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" Jul 10 01:10:51.165207 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.164794 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" event={"ID":"7b583dba-3f9b-4ed9-a08c-b138f7005fed","Type":"ContainerDied","Data":"b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da"} Jul 10 01:10:51.165207 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.164853 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n" event={"ID":"7b583dba-3f9b-4ed9-a08c-b138f7005fed","Type":"ContainerDied","Data":"29305aa594fb62c8c4ee26b9013670984b1f308a3c7246714accce893ac29d8c"} Jul 10 01:10:51.165207 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.164889 2579 scope.go:117] "RemoveContainer" containerID="b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da" Jul 10 01:10:51.173659 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.173643 2579 scope.go:117] "RemoveContainer" containerID="b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da" Jul 10 01:10:51.173943 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:10:51.173922 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da\": container with ID starting with b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da not found: ID does not exist" containerID="b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da" Jul 10 01:10:51.174039 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.173950 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da"} err="failed to get container status \"b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da\": rpc error: code = NotFound desc = could not find container \"b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da\": container with ID starting with b38ce07e53e07f4942a4537aec8861042d457983be5b9be0acb9300a074046da not found: ID does not exist" Jul 10 01:10:51.184610 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.184586 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n"] Jul 10 01:10:51.187035 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:51.187010 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-48a750-78c4684846-s4g7n"] Jul 10 01:10:52.025695 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:52.025658 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b583dba-3f9b-4ed9-a08c-b138f7005fed" path="/var/lib/kubelet/pods/7b583dba-3f9b-4ed9-a08c-b138f7005fed/volumes" Jul 10 01:10:57.267226 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.267190 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d"] Jul 10 01:10:57.267823 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.267468 2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" podUID="3407cca8-9d7f-4414-9b26-2ed27b54432f" containerName="maas-api" containerID="cri-o://eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2" gracePeriod=30 Jul 10 01:10:57.712778 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.712750 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:10:57.817037 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.816945 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3407cca8-9d7f-4414-9b26-2ed27b54432f-openshift-service-ca\") pod \"3407cca8-9d7f-4414-9b26-2ed27b54432f\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " Jul 10 01:10:57.817037 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.817014 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls\") pod \"3407cca8-9d7f-4414-9b26-2ed27b54432f\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " Jul 10 01:10:57.817268 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.817066 2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh5l2\" (UniqueName: \"kubernetes.io/projected/3407cca8-9d7f-4414-9b26-2ed27b54432f-kube-api-access-xh5l2\") pod \"3407cca8-9d7f-4414-9b26-2ed27b54432f\" (UID: \"3407cca8-9d7f-4414-9b26-2ed27b54432f\") " Jul 10 01:10:57.817351 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.817329 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3407cca8-9d7f-4414-9b26-2ed27b54432f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3407cca8-9d7f-4414-9b26-2ed27b54432f" (UID: "3407cca8-9d7f-4414-9b26-2ed27b54432f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 01:10:57.819358 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.819328 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3407cca8-9d7f-4414-9b26-2ed27b54432f" (UID: "3407cca8-9d7f-4414-9b26-2ed27b54432f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 01:10:57.819358 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.819335 2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3407cca8-9d7f-4414-9b26-2ed27b54432f-kube-api-access-xh5l2" (OuterVolumeSpecName: "kube-api-access-xh5l2") pod "3407cca8-9d7f-4414-9b26-2ed27b54432f" (UID: "3407cca8-9d7f-4414-9b26-2ed27b54432f"). InnerVolumeSpecName "kube-api-access-xh5l2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 01:10:57.918397 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.918359 2579 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3407cca8-9d7f-4414-9b26-2ed27b54432f-maas-api-tls\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:10:57.918397 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.918388 2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh5l2\" (UniqueName: \"kubernetes.io/projected/3407cca8-9d7f-4414-9b26-2ed27b54432f-kube-api-access-xh5l2\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:10:57.918397 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:57.918397 2579 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3407cca8-9d7f-4414-9b26-2ed27b54432f-openshift-service-ca\") on node \"ip-10-0-131-148.ec2.internal\" DevicePath \"\"" Jul 10 01:10:58.200982 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.200944 2579 generic.go:356] "Generic (PLEG): container finished" podID="3407cca8-9d7f-4414-9b26-2ed27b54432f" containerID="eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2" exitCode=0 Jul 10 01:10:58.201157 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.200994 2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" Jul 10 01:10:58.201157 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.200992 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" event={"ID":"3407cca8-9d7f-4414-9b26-2ed27b54432f","Type":"ContainerDied","Data":"eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2"} Jul 10 01:10:58.201157 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.201122 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d" event={"ID":"3407cca8-9d7f-4414-9b26-2ed27b54432f","Type":"ContainerDied","Data":"886be5197312bd9896ab4c35e07fb02fec9c4eabb58d9a797eccd669736be7cf"} Jul 10 01:10:58.201157 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.201152 2579 scope.go:117] "RemoveContainer" containerID="eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2" Jul 10 01:10:58.209225 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.209207 2579 scope.go:117] "RemoveContainer" containerID="eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2" Jul 10 01:10:58.209508 ip-10-0-131-148 kubenswrapper[2579]: E0710 01:10:58.209490 2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2\": container with ID starting with eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2 not found: ID does not exist" containerID="eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2" Jul 10 01:10:58.209576 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.209520 2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2"} err="failed to get container status \"eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2\": rpc error: code = NotFound desc = could not find container \"eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2\": container with ID starting with eb13ef717d3d289b5b7a18e1467917c109cefb2d5187d67d49072d2b8f1bfca2 not found: ID does not exist" Jul 10 01:10:58.216046 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.216019 2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d"] Jul 10 01:10:58.219977 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:10:58.219958 2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-aaea96-57dc69bdcc-qk49d"] Jul 10 01:11:00.027548 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:00.027515 2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3407cca8-9d7f-4414-9b26-2ed27b54432f" path="/var/lib/kubelet/pods/3407cca8-9d7f-4414-9b26-2ed27b54432f/volumes" Jul 10 01:11:09.151107 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:09.151083 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54ccd4f945-dclxs_bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48/copy-manifests/0.log" Jul 10 01:11:09.158519 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:09.158489 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54ccd4f945-dclxs_bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48/manager/0.log" Jul 10 01:11:09.758766 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:09.758738 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c5447fb99-fcplc_c5a020d6-f9a5-4e0a-8844-93373f810970/maas-api/0.log" Jul 10 01:11:09.874514 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:09.874484 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ptk2z_41848e61-3f3d-4558-9e42-0d5ad1a880e8/postgres/0.log" Jul 10 01:11:12.470517 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:12.470432 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbdhw_5261ad2f-3b2c-4d15-a04a-d609b3555ad7/discovery/0.log" Jul 10 01:11:13.057097 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:13.057062 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6pkcd_aa267b4e-c553-4807-8665-4c86b50b1c65/payload-processing/0.log" Jul 10 01:11:13.631344 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:13.631308 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-28hf7_9d177344-ce21-427b-b37d-aebab975a7bd/main/0.log" Jul 10 01:11:13.975336 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:13.975237 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-d5rpq_48d42cda-d7e3-4e44-b6bb-1d6d41b40965/main/0.log" Jul 10 01:11:20.100022 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:20.099989 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 01:11:20.106039 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:20.106018 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tjxpd_bbde56ff-c2db-4d64-886b-ff6d2554e16c/ovn-acl-logging/0.log" Jul 10 01:11:20.948973 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:20.948944 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6zh4c_7886b79d-8489-44af-9072-36341cd95e39/global-pull-secret-syncer/0.log" Jul 10 01:11:21.120282 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:21.120236 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hkfgl_b3358aa8-4e79-40b8-934a-c892dd63b988/konnectivity-agent/0.log" Jul 10 01:11:21.162381 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:21.162354 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-148.ec2.internal_24e0180dde756a21daf592d33e7c86a2/haproxy/0.log" Jul 10 01:11:28.332956 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:28.332923 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8nfhh_16a526ed-bb88-4551-9779-1d89ffd9cfaf/node-exporter/0.log" Jul 10 01:11:28.357401 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:28.357367 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8nfhh_16a526ed-bb88-4551-9779-1d89ffd9cfaf/kube-rbac-proxy/0.log" Jul 10 01:11:28.381669 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:28.381642 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8nfhh_16a526ed-bb88-4551-9779-1d89ffd9cfaf/init-textfile/0.log" Jul 10 01:11:31.306680 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.306643 2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78"] Jul 10 01:11:31.307064 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.306991 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b583dba-3f9b-4ed9-a08c-b138f7005fed" containerName="maas-api" Jul 10 01:11:31.307064 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.307006 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b583dba-3f9b-4ed9-a08c-b138f7005fed" containerName="maas-api" Jul 10 01:11:31.307064 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.307018 2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3407cca8-9d7f-4414-9b26-2ed27b54432f" containerName="maas-api" Jul 10 01:11:31.307064 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.307023 2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3407cca8-9d7f-4414-9b26-2ed27b54432f" containerName="maas-api" Jul 10 01:11:31.307241 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.307078 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b583dba-3f9b-4ed9-a08c-b138f7005fed" containerName="maas-api" Jul 10 01:11:31.307241 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.307085 2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3407cca8-9d7f-4414-9b26-2ed27b54432f" containerName="maas-api" Jul 10 01:11:31.309858 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.309839 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.312122 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.312100 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z5pqd\"/\"openshift-service-ca.crt\"" Jul 10 01:11:31.312911 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.312891 2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z5pqd\"/\"kube-root-ca.crt\"" Jul 10 01:11:31.313009 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.312891 2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z5pqd\"/\"default-dockercfg-dp87v\"" Jul 10 01:11:31.315939 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.315918 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78"] Jul 10 01:11:31.383621 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.383585 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvhh4\" (UniqueName: \"kubernetes.io/projected/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-kube-api-access-xvhh4\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.383799 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.383634 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-lib-modules\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.383799 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.383715 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-podres\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.383799 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.383746 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-proc\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.383799 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.383766 2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-sys\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484501 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484452 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-podres\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484501 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484504 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-proc\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484754 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484532 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-sys\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484754 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484571 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvhh4\" (UniqueName: \"kubernetes.io/projected/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-kube-api-access-xvhh4\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484754 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484611 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-proc\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484754 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484624 2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-lib-modules\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484754 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484646 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-podres\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484754 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484661 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-sys\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.484754 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.484721 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-lib-modules\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.491642 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.491618 2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvhh4\" (UniqueName: \"kubernetes.io/projected/3c1eb55e-9216-499c-bc8f-b75f64f49cfd-kube-api-access-xvhh4\") pod \"perf-node-gather-daemonset-mkn78\" (UID: \"3c1eb55e-9216-499c-bc8f-b75f64f49cfd\") " pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.620181 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.620144 2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:31.749949 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:31.749925 2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78"] Jul 10 01:11:31.752967 ip-10-0-131-148 kubenswrapper[2579]: W0710 01:11:31.752938 2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3c1eb55e_9216_499c_bc8f_b75f64f49cfd.slice/crio-2e458b96a9b5b5952d8a482859590e5edeca670284334d9c5536a196fe622ddb WatchSource:0}: Error finding container 2e458b96a9b5b5952d8a482859590e5edeca670284334d9c5536a196fe622ddb: Status 404 returned error can't find the container with id 2e458b96a9b5b5952d8a482859590e5edeca670284334d9c5536a196fe622ddb Jul 10 01:11:32.337771 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:32.337738 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" event={"ID":"3c1eb55e-9216-499c-bc8f-b75f64f49cfd","Type":"ContainerStarted","Data":"150062257f552eef948839970970bc66cbc83d98ca89365ee6ffe99065720fca"} Jul 10 01:11:32.338148 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:32.337790 2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" event={"ID":"3c1eb55e-9216-499c-bc8f-b75f64f49cfd","Type":"ContainerStarted","Data":"2e458b96a9b5b5952d8a482859590e5edeca670284334d9c5536a196fe622ddb"} Jul 10 01:11:32.338148 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:32.337878 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:32.352842 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:32.352773 2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" podStartSLOduration=1.352759963 podStartE2EDuration="1.352759963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 01:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 01:11:32.351123337 +0000 UTC m=+2712.793852412" watchObservedRunningTime="2026-07-10 01:11:32.352759963 +0000 UTC m=+2712.795489013" Jul 10 01:11:33.354535 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:33.354500 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lwpfk_ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8/dns/0.log" Jul 10 01:11:33.374439 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:33.374413 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lwpfk_ce31d46b-cb81-4d6c-8b9d-d6f4fbff58a8/kube-rbac-proxy/0.log" Jul 10 01:11:33.445030 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:33.445002 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xtbwc_07492d99-1fb7-4079-ac5a-1c9e41d1ed5c/dns-node-resolver/0.log" Jul 10 01:11:34.039160 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:34.039104 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmzlh_655a9d60-5c3a-455b-832e-56e8a1062561/node-ca/0.log" Jul 10 01:11:35.174530 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:35.174487 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbdhw_5261ad2f-3b2c-4d15-a04a-d609b3555ad7/discovery/0.log" Jul 10 01:11:35.353736 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:35.353702 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6pkcd_aa267b4e-c553-4807-8665-4c86b50b1c65/payload-processing/0.log" Jul 10 01:11:36.131751 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:36.131715 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pm99w_59844dc5-d391-4c33-8851-e79dd39ffb86/serve-healthcheck-canary/0.log" Jul 10 01:11:36.623920 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:36.623868 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kvzq5_c5a2d808-e0e7-475a-b3d6-a04be12797e5/kube-rbac-proxy/0.log" Jul 10 01:11:36.647710 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:36.647681 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kvzq5_c5a2d808-e0e7-475a-b3d6-a04be12797e5/exporter/0.log" Jul 10 01:11:36.669974 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:36.669946 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kvzq5_c5a2d808-e0e7-475a-b3d6-a04be12797e5/extractor/0.log" Jul 10 01:11:38.351018 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:38.350986 2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z5pqd/perf-node-gather-daemonset-mkn78" Jul 10 01:11:39.233958 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:39.233924 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54ccd4f945-dclxs_bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48/manager/0.log" Jul 10 01:11:39.253460 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:39.253432 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54ccd4f945-dclxs_bdf8c998-b03d-4e9f-a3bf-b2bafc2fdb48/copy-manifests/0.log" Jul 10 01:11:40.839198 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:40.839168 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54495c74ff-5cstl_90b13924-06cf-43a0-bd14-6b5acc4f212f/manager/0.log" Jul 10 01:11:41.827361 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:41.827329 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbdhw_5261ad2f-3b2c-4d15-a04a-d609b3555ad7/discovery/0.log" Jul 10 01:11:41.952167 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:41.952131 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6pkcd_aa267b4e-c553-4807-8665-4c86b50b1c65/payload-processing/0.log" Jul 10 01:11:43.387041 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:43.387010 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gbdhw_5261ad2f-3b2c-4d15-a04a-d609b3555ad7/discovery/0.log" Jul 10 01:11:43.518116 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:43.518075 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6pkcd_aa267b4e-c553-4807-8665-4c86b50b1c65/payload-processing/0.log" Jul 10 01:11:47.537883 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.537809 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qfwkw_12ba7873-dbe9-43a9-9f47-86f4280640c6/kube-multus-additional-cni-plugins/0.log" Jul 10 01:11:47.560756 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.560731 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qfwkw_12ba7873-dbe9-43a9-9f47-86f4280640c6/egress-router-binary-copy/0.log" Jul 10 01:11:47.583980 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.583958 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qfwkw_12ba7873-dbe9-43a9-9f47-86f4280640c6/cni-plugins/0.log" Jul 10 01:11:47.605345 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.605318 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qfwkw_12ba7873-dbe9-43a9-9f47-86f4280640c6/bond-cni-plugin/0.log" Jul 10 01:11:47.629412 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.629383 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qfwkw_12ba7873-dbe9-43a9-9f47-86f4280640c6/routeoverride-cni/0.log" Jul 10 01:11:47.650089 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.650061 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qfwkw_12ba7873-dbe9-43a9-9f47-86f4280640c6/whereabouts-cni-bincopy/0.log" Jul 10 01:11:47.670942 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.670917 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qfwkw_12ba7873-dbe9-43a9-9f47-86f4280640c6/whereabouts-cni/0.log" Jul 10 01:11:47.930410 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:47.930384 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-m8ldd_853e3939-f02a-467a-844d-b22ecfd84736/kube-multus/0.log" Jul 10 01:11:48.036652 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:48.036620 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mqw4x_d5f71bf8-0d90-4b94-84c2-e706db351d5a/network-metrics-daemon/0.log" Jul 10 01:11:48.056964 ip-10-0-131-148 kubenswrapper[2579]: I0710 01:11:48.056929 2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mqw4x_d5f71bf8-0d90-4b94-84c2-e706db351d5a/kube-rbac-proxy/0.log"